Convolution Kernels by Hand
Nine numbers decide whether an image blurs, sharpens or turns into an outline. Edit them and watch which.
Overview
One operation, many effects
Convolution is a single, simple operation that produces a startling range of results depending on nine numbers. Take a small grid of weights — the kernel — centre it on a pixel, multiply each weight by the pixel underneath it, and add the products together. That sum is the output pixel. Slide the kernel one step and do it again.
That is the whole algorithm. Blurring, sharpening, edge detection and embossing are not different algorithms; they are the same algorithm with different numbers in the grid.
Convolution Kernels by Hand
Worth knowing
Sobel X and Sobel Y are the same kernel rotated. Each finds edges running across it, not along it.Convolution Kernels by Hand
Nine numbers, one sliding window, and most of classical image processing.
Reading a kernel
The identity kernel is the easiest to reason about:
0 0 0
0 1 0
0 0 0Every neighbour is multiplied by zero, the centre pixel by one, so the output is the input unchanged. Now change the centre to 5 and set the four orthogonal neighbours to −1:
0 -1 0
-1 5 -1
0 -1 0The centre pixel is amplified and its neighbours are subtracted from it. Where the neighbourhood is flat, the subtraction cancels the amplification exactly and nothing happens. Where the centre differs from its surroundings — at an edge — the difference is exaggerated. That is sharpening, and it explains why sharpening amplifies noise: a noisy pixel is, by definition, a pixel that differs from its neighbours.
The sum tells you what kind of filter it is
There are two families, and you can tell them apart by adding the weights up.
Weights summing to 1 preserve average brightness. A box blur of nine ones divided by nine is the arithmetic mean of the neighbourhood; a Gaussian blur weights the centre more heavily than the corners. Both keep the image at the same overall exposure because the total contribution of every pixel is unchanged.
Weights summing to 0 produce a difference image. Flat regions become zero, because a constant multiplied by weights that cancel gives nothing. Only where the image changes does anything survive. Since a result can be negative, these filters are usually displayed with 128 added, which is why edge images have that flat grey background.
| Kernel | Sum | What survives |
|---|---|---|
| Identity | 1 | everything, unchanged |
| Box blur | 1 (after division) | low-frequency detail |
| Sharpen | 1 | everything, with edges exaggerated |
| Sobel | 0 | edges in one direction |
| Laplacian | 0 | edges in every direction |
| Emboss | 1 | a directional shadow |
Sobel: why there are two of them
The Sobel operator comes as a pair, and the pair is one kernel rotated by ninety degrees:
Sobel X Sobel Y
-1 0 1 -1 -2 -1
-2 0 2 0 0 0
-1 0 1 1 2 1Sobel X subtracts the column on the left from the column on the right. A vertical edge — where left and right differ — produces a large value. A horizontal edge produces nothing at all, because left and right are identical there. Sobel Y is the same argument with rows.
Run each on the visualisation above and the asymmetry is obvious: the vertical sides of the rectangle appear under Sobel X and vanish under Sobel Y. In practice the two are combined, usually as the square root of the sum of their squares, to get an edge strength that does not care about direction.
The middle row being −2, 0, 2 rather than −1, 0, 1 is a small piece of smoothing built in: the pixel in line with the centre is weighted more than the diagonals, so a single noisy pixel has less influence than it would in a pure difference.
What happens at the border
A kernel centred on a corner pixel has five of its nine cells hanging over the edge of the image. Something has to be decided, and the three common answers are to leave the border out (producing a smaller output), to treat the missing pixels as zero, or to repeat the nearest real pixel outwards. The visualisation here repeats the edge pixel, which is why the frame stays clean; zero-padding would darken it. The [padding module](padding_in_cnn.html) works through what that choice costs inside a network.
Why this matters for CNNs
Everything above is fixed: someone chose the nine numbers and wrote them down. A convolutional layer in a neural network performs the identical operation and differs in exactly one respect — the nine numbers are parameters, and gradient descent chooses them.
That is worth sitting with. When people say a CNN "learns to detect edges in the first layer", they mean that training drives some of the learned kernels towards something that looks very much like Sobel, because edges turn out to be a useful thing to measure. Nobody put Sobel there. The architecture only supplies the sliding window; the content of the window is discovered.
One output pixel, worked
A convolution is a weighted sum repeated at every position, and the character of a kernel is legible from its numbers. Take the sharpen kernel — centre 5, the four orthogonal neighbours −1, corners 0 — and apply it where a centre pixel of 100 sits among neighbours of 90:
5×100 − 90 − 90 − 90 − 90 = 140
| patch | sharpen output |
|---|---|
| centre 100, neighbours 90 (an edge) | 140 — the difference exaggerated |
| centre 100, all neighbours 100 (flat) | 100 — unchanged |
On flat ground the amplification of the centre and the subtraction of the neighbours cancel exactly, so nothing happens; only where the centre differs from its surroundings does the output diverge, which is why sharpening exaggerates edges and, for the same reason, amplifies noise — a noisy pixel *is* one that differs from its neighbours. The kernel's weight sum tells you its family: sum to 1 (identity, blur, sharpen) preserves brightness, sum to 0 (Sobel, Laplacian) produces a difference image that is flat-grey where nothing changes. A CNN's first layer runs exactly this operation with the nine numbers as *learned* parameters — and training reliably drives some of them toward something very like Sobel, because edges turn out to be worth measuring, though nobody put Sobel there.
Where it goes wrong
Sharpening noise. Any kernel with negative weights amplifies pixel-to-pixel variation, and noise is pixel-to-pixel variation. Denoise first, sharpen second.
Forgetting to normalise. A blur kernel of nine ones, applied without dividing by nine, produces an image nine times too bright — which, after clipping at 255, is a white rectangle.
Assuming bigger is better. A 3×3 kernel sees a 3×3 region. To see further you either use a larger kernel, which costs quadratically more multiplications, or stack several small ones, which is what modern architectures do and why [receptive field](feature_map_in_cnn.html) is a concept worth having.
Slide a 3x3 grid over an image and watch what each one finds
A convolution is a weighted sum repeated at every position. Six classic kernels are applied to the same small image here, with the arithmetic for one output pixel written out in full.
Check yourself
0 of …Answer without scrolling back up.
A kernel's nine weights sum to zero. What does its output look like on a flat, uniform region?
Why does Sobel X find vertical edges rather than horizontal ones?
What is the only real difference between this and a CNN's convolutional layer?
Convolution Kernels by Hand
Convolution is a single, simple operation that produces a startling range of results depending on nine numbers. Take a small grid of weights — the kernel — centre it on a pixel, multiply each weight by the pixel underneath it, and add the products together. That sum is the output pixel. Slide the kernel one step and do it again.