Skip to main content

Convolutional Neural Networks

A 32×32 color image holds 3×32×32=30723\times32\times32=3072 numbers. Turning it into 16 feature maps of 16×16 (16×16×16=409616\times16\times16=4096 numbers) with a fully connected layer means connecting every output to every input: 3072×4096≈12.63072\times4096\approx12.6 million weights, and the count keeps growing as the image gets larger. A convolutional neural network (CNN) performs the same change of shape with 3×3 kernels and only 448 parameters (computed below), whatever the image size. It saves that many parameters by building two assumptions about grid-like data, such as images, into the layer:

  • locality: nearby values interact before distant values;
  • parameter sharing: the same learned detector is applied across positions.

A Kernel Computes the Same Weighted Sum at Every Position​

Deep-learning libraries usually implement cross-correlation even when the operation is called convolution: the kernel covers a small patch of the input, multiplies matching entries and adds them up, then slides to the next position and repeats. For input X\mathbf{X} and kernel V\mathbf{V},

Hi,j,d=bd+∑a∑b∑cVa,b,c,d Xi+a,j+b,c.H_{i,j,d} = b_d + \sum_a\sum_b\sum_c V_{a,b,c,d}\,X_{i+a,j+b,c}.

Here (i,j)(i,j) is the output position, (a,b)(a,b) moves over the kernel's height and width, and cc runs over input channels. One kernel spans all input channels cc and produces one output channel dd; multiple learned kernels create multiple feature maps.

For one spatial dimension, input length nn, kernel width kk, padding pp on each side, and stride ss, the output length is

⌊n+2p−ks⌋+1.\left\lfloor\frac{n+2p-k}{s}\right\rfloor+1.

In words: the padded length minus the kernel width is how far the kernel can slide; count how many stops the stride allows, then add the starting position. The same calculation applies independently to height and width.

A 2 by 2 kernel applied to a 3 by 3 input produces a 2 by 2 output; the highlighted window produces 19.Open full-size image

The shaded window uses the same kernel entries at every position. Its first output is 0×0 + 1×1 + 3×2 + 4×3 = 19. Sliding the window produces the remaining outputs. This is a single-channel, stride-one example without padding.

What a convolution computes
Kernel

Input image 10×10

1111111111119911999111991199911199119991119911111111991111111199111111119999991111999999111111111111

Feature map 8×8 · Vertical edge

1616−16−1616160−162424−24−2424240−242424−24−2416160−162424−24−24880−82424−24−2400002424−16−1600−8−82424−8−800−16−1616160000−16−16

row 1, column 1sum = 16

The same 9 weights are used at every position. Each feature-map cell is the 9 products of pixel × weight added together (bias 0).

Compute a Kernel and a Shape​

For input [1,2,4,8][1,2,4,8], kernel [1,−1][1,-1], zero bias, no padding, and stride one, cross-correlation produces [−1,−2,−4][-1,-2,-4]: each output subtracts adjacent values. Mathematical convolution reverses the kernel and would produce [1,2,4][1,2,4] under the same valid-window convention.

For an ordinary dense 2-D layer, input [B, 3, 32, 32] and 16 kernels of shape [3, 3, 3] (input channels, height, width), with padding one and stride two, give output [B, 16, 16, 16]: the output-length formula gives ⌊(32+2−3)/2⌋+1=16\lfloor(32+2-3)/2\rfloor+1=16. Each kernel has 3⋅3⋅3=273\cdot3\cdot3=27 weights and one bias, so the 16 kernels hold 16(3⋅3⋅3+1)=44816(3\cdot3\cdot3+1)=448 learned parameters, independent of image width and height; this is the number compared with the 12.6 million fully connected weights above. The shared weight gradient sums contributions from every position and batch example that used that weight.

Shift the Input and the Output Shifts Too​

Parameter sharing makes the representation translation equivariant: shifting the input shifts the resulting feature map. Take the 1-D example again and put a 0 in front of the input, giving [0,1,2,4,8][0,1,2,4,8]. The output becomes [−1,−1,−2,−4][-1,-1,-2,-4]. The original [−1,−2,−4][-1,-2,-4] has moved one place to the right, and the extra −1-1 at the front comes from the new boundary value. That is equivariance.

Equivariance is not invariance: the output moves along with the input instead of staying the same. Pooling, aggregation, augmentation, and task design may produce some tolerance to shifts, but convolution alone does not make the output unchanged, and pooling does not guarantee invariance to arbitrary shifts.

Exact equivariance holds only on an infinite grid with stride one: finite padding breaks it at edges, and stride-two sampling generally preserves only shifts aligned with the sampling grid. As the padding and stride discussion illustrates, boundary treatment matters. The displayed cross-correlation equation assumes stride one, dilation one, and valid indices; padding supplies additional boundary values.

More Layers Let Each Output See a Larger Region​

The receptive field is the region of the input that ultimately affects one output point. An output of one 3×3 convolution sees a 3×3 patch of the input. Stack a second 3×3 layer with stride one: each point now sees 3×3 points of the first layer, and those points together cover 5×5 of the input, so two stacked layers have a 5×5 receptive field.

Dilation adds spacing between kernel elements, enlarging the receptive field without adding parameters, and changes the effective kernel size: with dilation rr, replace kk by r(k−1)+1r(k-1)+1 in the output-length formula. A width-3 kernel with r=2r=2, for example, has effective width 2⋅2+1=52\cdot2+1=5.

Building Blocks Control Size, Channels, and Reach​

  • Padding controls border treatment and often preserves spatial size.
  • Stride subsamples while applying the kernel.
  • Pooling summarizes local neighborhoods, by maximum or mean, without learned spatial weights.
  • 1×11\times1 convolution mixes channels independently at each spatial position and can change the channel count; adding nonlinearity still needs an activation.
  • Stacked layers increase the receptive field and compose local features into higher-level representations.

When the Data Breaks These Assumptions, a CNN Loses Its Advantage​

Downsampling discards small objects. A 4×4-pixel part shrinks to 2×2 after one 2×2 pooling with stride two and to a single pixel after a second, losing its shape and precise location. In the same way, locality and translation structure are not universal, so the useful inductive bias depends on the data; a large receptive field does not prove that the model effectively uses all relevant context; accuracy under random crops does not establish robustness to real distribution shift; and dataset construction and augmentation choices can dominate architecture changes.

Use a small MLP or linear model as a baseline when the input does not clearly benefit from spatial structure. For maintained implementations and exercises, see Dive into Deep Learning: Convolutional Neural Networks.

CNN Explainer lets you follow an example image through a small convolutional network. Inspect a convolution and its activation map, then compare what remains after pooling. This connects the local kernel calculation above to the changing spatial size and channel count across a full network.

Explore connectionsOpen network