Resizing and Interpolation
Enlarging an image means inventing pixels. How you invent them is the difference between blocks and mush.
Overview
Enlarging invents; shrinking discards
Resizing is two different problems wearing one name.
Enlarging asks for pixels that were never measured. A 4×4 image stretched to 8×8 needs 64 values where 16 exist. The extra 48 have to come from somewhere, and wherever they come from, they are a guess. No interpolation method adds information — the choice is only about which plausible guess to make.
Shrinking has too much data rather than too little, and the risk is throwing away the wrong parts.
Resizing and Interpolation
Worth knowing
Resizing and Interpolation
What happens between the pixels, and why the answer looks different every time.
Nearest neighbour
For each output pixel, work out where it lands in the input and copy whichever input pixel is closest. That is all.
It is the fastest possible method, and it is exact in one important sense: every output value is a value that genuinely appeared in the input. Nothing is averaged, so nothing is invented that was not there.
The cost is blockiness. Neighbouring output pixels that map to the same input pixel are identical, so enlarging produces flat squares with hard steps between them. Diagonal edges come out as staircases.
That is often exactly right. Pixel art must be scaled with nearest neighbour or it stops being pixel art. A segmentation mask where pixel value 3 means "road" must be scaled with nearest neighbour, because averaging 3 and 5 gives 4, which might mean "building" — a label is a name, not a quantity, and names do not average.
Bilinear
For each output pixel, find the four input pixels surrounding its position and take a weighted average, with weights from how close the position is to each. Linear in one direction, then linear in the other; hence bilinear.
Diagonals become smooth, gradients stay gradual, and the staircase disappears. The cost is softness: an average of neighbours is by definition less extreme than its inputs, so hard edges are pulled apart into ramps. Enlarge by a large factor and the result is not blocky but mushy.
Switch between the two above with the downscale factor set high. The same information is present in both; only the guessing strategy differs, and it is easy to see that neither is simply better.
Bicubic extends the idea to sixteen surrounding pixels with a cubic weight function. It is sharper than bilinear because the cubic curve overshoots slightly at edges, adding a small amount of local contrast. Photo editors default to it for that reason. The overshoot is also its failure mode: it can produce faint halos beside high-contrast edges.
| Method | Reads | Speed | Good for | Fails at |
|---|---|---|---|---|
| Nearest | 1 pixel | fastest | pixel art, label masks | photographs |
| Bilinear | 4 pixels | fast | general enlargement | large factors |
| Bicubic | 16 pixels | slower | photographs | can halo |
| Area | a block | fast | shrinking | not for enlarging |
Shrinking is a different failure
To halve an image, it is tempting to keep every second pixel. This is wrong, and wrong in a way that looks like a bug elsewhere in the program.
Consider a pattern of alternating black and white columns. Keep every second column and you may get all the black ones — a solid black image. Shift by one and you get solid white. Detail finer than the new sampling grid does not disappear gracefully; it reappears as a completely different, false pattern. This is aliasing, and it is why photographs of striped shirts on video develop rainbow moiré.
The fix is to average before sampling. Area interpolation takes the mean of every input pixel that falls inside each output pixel's footprint, so nothing is silently dropped. Gaussian blurring before downsampling achieves the same end, and is what an image pyramid does at every level.
This matters for machine learning specifically. A dataset resized with a naive-sampling routine has aliased artefacts that vary with the original resolution, and a network will happily learn them.
Bilinear, worked, and why masks forbid it
Bilinear interpolation reads the four input pixels around a fractional position and takes a distance-weighted average — linear across, then linear down. For an output point landing at fraction (0.25, 0.75) inside a cell whose corners are 10 (top-left), 20 (top-right), 30 (bottom-left), 40 (bottom-right):
| step | value |
|---|---|
| top edge: 10·0.75 + 20·0.25 | 12.5 |
| bottom edge: 30·0.75 + 40·0.25 | 32.5 |
| down: 12.5·0.25 + 32.5·0.75 | 27.5 |
The output, 27.5, is a value that appears nowhere in the input — and that is the whole point for a photograph, where it smooths the staircase nearest-neighbour would leave. It is also exactly why a *label mask* must never be resized this way: if 3 means "road" and 5 means "building", their bilinear average of 4 might mean something else entirely, because a class index is a name, not a quantity, and names do not average. Nearest neighbour, which copies the single closest pixel, is the only correct choice there. Shrinking is a third case again: naively keeping every second pixel aliases — a striped pattern can vanish or invert — so it needs an average (area interpolation) before sampling, not an interpolation between samples.
Bicubic extends the same idea to sixteen surrounding pixels with a cubic weight that overshoots slightly at edges, adding a little local contrast — which is why photo editors default to it and also why it can leave faint halos beside high-contrast edges. Shrinking is the failure people meet as a mysterious bug elsewhere: keeping every second pixel of an alternating pattern can return a solid field of one colour, because detail finer than the new sampling grid reappears as a false pattern — aliasing, the same effect as moiré on a striped shirt on video. The cure is to average before sampling (area interpolation, or a Gaussian blur first, which is what an image pyramid does), and it matters for machine learning specifically, because a dataset resized by naive sampling carries artefacts that vary with the original resolution and a network will happily learn them.
Where it goes wrong
Bilinear on a label mask. Averaging class indices produces indices that mean something else. Nearest neighbour, always.
Enlarging to fix a low-resolution input. Upscaling before feeding a model adds no information and costs computation. Learned super-resolution is a different thing entirely — it hallucinates plausible detail from a prior, which is useful for viewing and dangerous for measurement.
Repeated resizes. Each interpolation softens. Resize once from the original rather than in steps.
Ignoring aspect ratio. Stretching to a square distorts every shape in the frame. Pad to the target aspect ratio, then resize.
Nearest, bilinear, and why downscaling needs a blur first
Resizing asks for pixel values at coordinates that do not exist, so every method is a guess. This measures three guesses against a known ground truth, and then shows the failure that catches everyone -- that shrinking an image correctly is not an interpolation problem at all.
Check yourself
0 of …Answer without scrolling back up.
Why must a segmentation mask be resized with nearest neighbour?
What causes aliasing when shrinking an image?
Why can bicubic interpolation produce halos near high-contrast edges?
Resizing and Interpolation
Enlarging asks for pixels that were never measured. A 4×4 image stretched to 8×8 needs 64 values where 16 exist. The extra 48 have to come from somewhere, and wherever they come from, they are a guess. No interpolation method adds information — the choice is only about which plausible guess to make.