Convolution Operation

By the end of this lesson, you will be able to...

• compute the values and output shape of a small two-dimensional convolution;

• explain how local receptive fields and shared weights change the parameter budget of an image model;

• distinguish translation equivariance from the stronger claim of position invariance.

Idea in one sentence: A convolution reuses one small learned detector at every image location, producing a map of where that local pattern appears.

Core Insight

Suppose a warehouse camera photographs packages on a moving belt. The classifier must notice a short diagonal tear in a label. Today the tear is near the top-left; tomorrow the box is slightly shifted and the same tear is near the center.

A fully connected layer begins with a tempting model: flatten the image and give every pixel location its own connection. It can represent the problem, but it treats “diagonal tear at row 4, column 5” and “the same tear at row 24, column 25” as unrelated evidence. It must spend separate parameters learning the same visual fact twice.

The image gives us evidence for a better assumption: nearby pixels form meaningful small patterns, and a pattern worth detecting in one place is often worth detecting elsewhere. A convolutional layer encodes that assumption. It looks through a small window, applies the same weights at every allowed location, and records the result in a feature map.

That is an architectural bias, not magic. The bias is valuable when local repeated structure matters; it can be a poor fit when the decisive relation is arbitrary and global. First we will make the calculation visible, then name exactly what it buys.

The Small Situation: One Detector, Four Places

To keep the arithmetic inspectable, suppose the camera has already produced this single-channel 3 × 3 patch. The numbers are illustrative pixel intensities after a simple preprocessing step.

image X                    detector K
1  2  0                    1  0
0  3  1                    0  1
2  2  2

K is a 2 × 2 learned detector. Its positive weights reward a bright top-left to bottom-right diagonal; its zero weights ignore the other two cells. A real model learns these weights from data. Here we choose them so that we can inspect the mechanism.

With no padding and stride 1, the detector can start at four positions. Each position sees a 2 × 2 local patch. It does not see the full image at once.

This local window is the detector's receptive field for this layer. Locality is a teaching model for the common visual regularity that close pixels often combine into edges, textures, or small parts. It is not a claim that distant pixels can never matter; later layers can combine many local results.

The Initial Model: Different Weights for Every Location

If we flatten a 64 × 64 grayscale image, we have 4,096 input values. Asking for 32 outputs from a dense layer needs 4,096 × 32 = 131,072 weights before biases. More importantly, a weight associated with the top-left pixel is not automatically reused for a matching patch elsewhere.

That initial model works when absolute position is truly part of the task. For example, a fixed sensor whose upper-left cell has a special physical meaning may need location-specific parameters. It becomes wasteful for the package camera when a scratch can occur anywhere on the label.

The missing idea is not merely “use fewer parameters.” We want a detector whose meaning stays stable as it moves. The same numbers in K should score every local patch. Weight sharing gives both a smaller parameter budget and a structured way to report where the evidence occurred.

From a Local Dot Product to a Feature Map

Plain meaning: compare a small detector with a small piece of the input, multiply matching cells, and add the results.

In our package image: place K over the upper-left patch of X.

Technical name: deep-learning libraries call this operation convolution, although their usual implementation is mathematically cross-correlation because it does not flip the kernel. The useful learning mechanism here is the same: a learned local weighted sum slides across the input. PyTorch documents Conv2d using this cross-correlation form.

For the first location, the calculation is:

patch                         K                    output
1  2                          1  0
0  3              elementwise 0  1       sum = 1 + 0 + 0 + 3 = 4

The second location uses a different patch but exactly the same K:

patch                         K                    output
2  0                          1  0
3  1              elementwise 0  1       sum = 2 + 0 + 0 + 1 = 3

Repeating that operation produces the whole feature map:

feature map Y
4  3
2  5

The high value 5 says that, under this illustrative detector, the bottom-right patch matches the diagonal pattern more strongly than the other locations. It is not a class prediction. It is one intermediate piece of evidence. A layer normally has many learned kernels, so it produces many feature maps: one channel per detector.

So far, we have seen that a convolution is repeated local arithmetic with shared parameters. This matters because the output keeps the spatial arrangement of the detector's responses instead of immediately collapsing the image into one unstructured vector.

Reading the Shape Before Running the Code

For one input channel and one output channel, a valid H × W input with a kH × kW kernel, padding pH, pW, and stride sH, sW has output dimensions:

out_height = floor((H + 2pH - kH) / sH) + 1
out_width  = floor((W + 2pW - kW) / sW) + 1

Our example has H = W = 3, kH = kW = 2, p = 0, and s = 1. Therefore each output dimension is floor((3 - 2) / 1) + 1 = 2, matching the 2 × 2 table above.

Now give the camera an RGB batch. In the usual PyTorch layout, the input is (batch, in_channels, height, width). A Conv2d(3, 32, kernel_size=3, padding=1) has 32 kernels, each spanning all three input channels. With a batch of 16 64 × 64 images, its output is (16, 32, 64, 64).

conv = nn.Conv2d(in_channels=3, out_channels=32, kernel_size=3, padding=1)
x = torch.randn(16, 3, 64, 64)
y = conv(x)  # shape: (16, 32, 64, 64)

Padding 1 supplies a one-cell border for this 3 × 3 kernel, so the stride-1 layer can place a center on every original location. Padding values are ordinarily zero unless a different mode is chosen. Stride 2 would skip every other start position and reduce the spatial dimensions; the next lesson examines that downsampling trade-off.

What Moves When the Tear Moves?

Suppose the diagonal tear moves one pixel right, without crossing a border and with the same preprocessing. The relevant local patch appears one position right. Since the detector weights are shared, the strong response also moves one position right in the feature map.

tear moves right in X  ->  matching activation moves right in Y

This is translation equivariance: transforming the input by a small translation transforms the feature map in the corresponding way. It is not translation invariance, which would mean the final answer remains unchanged regardless of position.

Why is the distinction useful? A defect-localization task may need equivariance: the response must move so the model can say where the defect is. A package-level classification task often wants some later tolerance to position, but it earns that through training data, pooling or strided layers, broader receptive fields, aggregation, and the task objective. A single convolution proves none of those by itself.

What This Changes in the Model

Weight sharing changes the parameter count sharply. One 3 × 3 kernel for an RGB input has 3 × 3 × 3 = 27 weights, plus one bias if used. Thirty-two output channels therefore use 32 × (27 + 1) = 896 parameters. The same 32 detectors are applied at every spatial location.

That count is independent of image height and width. The compute is not: a larger image creates more output positions, so the layer performs more local dot products. Parameter efficiency does not mean a convolution is free.

This also explains why the feature-map channel dimension grows. Channels do not mean more pixels. They mean different learned questions asked at each spatial position: perhaps one detector responds to a diagonal boundary, another to a horizontal texture, and another to a color transition. The values only gain meaning from their learned weights and the task; we should not assume every early channel corresponds neatly to a human-named feature.

Costs, Limits, and Signals

Convolution helps when nearby repeated patterns are useful and the same detector should work in many places. It costs repeated computation over all locations and builds in a preference that may not fit the data. A small kernel cannot directly compare two far-apart points in one layer; stacked layers enlarge the effective receptive field, but that adds depth, computation, and optimization choices.

It also does not guarantee robustness to rotation, scale change, different illumination, unusual backgrounds, or a distribution shift in the warehouse camera. Those are empirical questions. Data augmentation may improve tolerance to a named variation, but it does not make every variation irrelevant.

Useful inspection signals are concrete:

• a shape ledger catches an unintended spatial collapse or channel mismatch;
• an activation map reveals whether a detector responds mainly at borders or everywhere;
• a controlled one-pixel shift tests the expected movement of intermediate responses;
• validation examples with shifted, rotated, or differently lit labels test the robustness the deployed task actually needs.

The boundary is visible when the model's shared local assumption stops matching the job. If the label's text at one corner must be compared with a seal at a distant corner, a single local detector is insufficient; the architecture needs a way to combine evidence over distance.

Common Confusions

Confusion: “A convolution only sees tiny patches, so the network can never use global context.”

Why it is tempting: one kernel is local. Better model: one layer is local; later layers combine neighboring feature-map values and enlarge the effective receptive field.

Confusion: “Moving an object does not change a convolutional output.”

Why it is tempting: weight sharing is often described as position robustness. Better model: the feature response usually moves with the object. That is equivariance; invariance requires additional decisions.

Confusion: “More output channels mean a higher-resolution image.”

Why it is tempting: both are tensor dimensions. Better model: height and width say where evidence occurs; output channels say how many learned detectors report at each location.

Check Your Understanding

Check: A 5 × 5 single-channel input uses one 3 × 3 kernel, no padding, and stride 1. What is the output shape?

Think first, then reveal.

Answer: 3 × 3. Each dimension is floor((5 - 3) / 1) + 1 = 3. The kernel has three valid starting positions along each axis.

Check: The same small scratch moves two pixels right, and a feature-map peak moves two cells right. Did the detector become position-invariant?

Answer: No. The movement of the response is evidence of equivariance. The detector still records a different spatial location; a later stage would need to make a position-tolerant decision if that is the goal.

Practice: Inspect a New Detector

A grayscale 4 × 4 image is processed by one 2 × 2 kernel with stride 2 and no padding. The kernel has one learned weight for each cell in its local window.

1. Predict the output shape.
2. Explain how many times the same four kernel weights are reused.
3. A vertical scratch shifts one pixel to the right. Is it safe to predict that a feature-map value will move exactly one output cell right?

Model answer: The output is 2 × 2: valid starts occur at rows 0 and 2 and columns 0 and 2. The same four weights are reused at four output locations. It is not safe to promise an exact one-cell shift because stride 2 samples starts only every two input cells; the scratch can land in a different receptive field or affect the same one. The architecture remains locally shared, but stride changes the spatial sampling. This is why shape and stride belong in the reasoning, not only in framework code.

Resources

• [DOCUMENTATION] PyTorch: Conv2d — Focus: the input/output shape convention, stride, padding, groups, and the cross-correlation equation used by the layer.
• [BOOK/TUTORIAL] Dive into Deep Learning: From Fully Connected Layers to Convolutions — Focus: why locality and translation equivariance motivate convolutional layers.
• [BOOK/TUTORIAL] Dive into Deep Learning: Convolutions for Images — Focus: hand-computing a small cross-correlation before using a framework.
• [PAPER] Gradient-Based Learning Applied to Document Recognition — Focus: the early convolution-and-subsampling design behind LeNet-5.

Key Takeaways

• A convolution produces a feature map by applying one local weighted sum at many locations with the same learned weights.
• Locality and weight sharing reduce parameters and preserve the location of a detected pattern, but they are assumptions about the data rather than universal advantages.
• A shifted input usually causes a shifted feature response: that is translation equivariance, not a guarantee that the final model ignores position.
• Kernel size, padding, stride, input channels, and output channels jointly determine the calculation, the output shape, and what the next layer can inspect.