Classic CNN Architectures
LESSON
Classic CNN Architectures
By the end of this lesson, you will be able to...
explain the design pressure behind LeNet, AlexNet, VGG, Inception, and ResNet;
compare regular depth, multi-branch computation, and residual connections by the constraint each addresses;
choose a useful architectural starting point for a named constraint without treating historical names as a performance ranking.
Idea in one sentence: A classic CNN is useful to remember when it shows a concrete design response to a bottleneck in scale, compute, or trainability.
Core Insight
Suppose a plant-inspection team has a working CNN for classifying leaves as healthy or diseased. The model is a simple sequence of convolution, activation, pooling, and a classifier head. It learns obvious spots but misses subtle patterns. A teammate proposes, “Let’s use the newest-sounding architecture,” while another says, “Just add more layers.”
Both suggestions hide the decision that matters. What is actually limiting the model? It might need more compositional depth. It might be spending too much compute at one resolution. Or it might already be deep enough that the added layers are difficult to optimize.
Classic CNNs are a compact record of teams facing versions of those pressures. They are not a museum of names and not a universal leaderboard. Each architecture makes one structural move and accepts a cost. Read the move, the pressure, and the evidence—not just the diagram.
The Promise We Need to Keep
The plant-inspection team needs a model that can classify one leaf image on its available hardware. Its promise is classification, not pixel-level segmentation and not a claim to invent a new vision backbone.
That means the first model should be inspectable. The team needs to explain its shape ledger, test whether leaf position or scale changes affect accuracy, and compare a few bounded alternatives. A more elaborate architecture is useful only if it addresses a measured limitation under their latency, memory, data, and maintenance constraints.
This is a situated preference: a regular baseline is often easier to debug when data and compute are limited. It is not a rule that every serious vision system should start with an old CNN, nor a claim that a named historical model is automatically the best modern deployment choice.
The Initial Model: More Layers Must Be Better
The team begins with a reasonable thought: deeper layers can combine small visual features into larger structures. A shallow detector may see a brown spot; several layers can combine spots, texture, and vein shape into evidence for disease.
This model works while the extra layers train and fit the resource budget. It breaks in two different ways:
- a uniform deeper stack can become expensive or allocate compute poorly across feature scales;
- even when a deeper stack is expressive enough in principle, optimization can become harder, so the training process fails to use the extra capacity well.
The missing model is that architecture is both a representation design and a path through which gradients and information travel. Different classic CNNs respond to different parts of that problem.
A Timeline of Pressures, Not a List of Names
The following table is a teaching model. Real papers contain more details than one row can hold, and each family has variants. The useful question is whether the row helps predict a design trade-off.
| Family | Pressure made visible | Structural move | What it buys | Boundary or cost |
|---|---|---|---|---|
| LeNet-5 | Can learned local features and downsampling form an end-to-end image classifier? | convolution, subsampling, and classification stages | a clear hierarchy from local patterns to a decision | designed for a much smaller image and compute setting than many later tasks |
| AlexNet | Can CNNs train effectively on a large visual-recognition task? | a deeper, high-capacity convolutional system with ReLUs, regularization, augmentation, and GPU training | showed a workable large-scale recipe in its setting | substantial data and compute needs; the ingredients matter together |
| VGG | How can depth be increased with a regular, understandable block pattern? | repeated small 3 × 3 convolutions |
simple stage structure and greater depth | many activations and parameters can be expensive |
| Inception | How can a network use multiple spatial scales without spending all compute in one uniform path? | parallel branches, with cheap projections used to manage expensive branches | more deliberate compute allocation and multi-scale processing | module shape and branch choices are more complex to inspect |
| ResNet | Why can deeper plain stacks become harder to train? | an identity shortcut plus a learned residual correction | an easier optimization path for substantially deeper networks | does not remove the need for data, validation, or a suitable stage design |
The table gives a strong reading habit: architecture names are shorthand for a pressure plus a move. If the team cannot name the pressure, copying the move is mostly cargo cult.
A Worked Design Review
The team runs a controlled baseline. Its validation split contains three named slices: common leaf images, leaves shifted in the frame, and small disease spots. The baseline is accurate enough on common images but misses many small spots. Its latency still has room to grow.
They consider three changes.
Option A: a VGG-like regular block
conv 3 × 3 -> activation -> conv 3 × 3 -> activation -> downsample
This keeps the architecture easy to trace. Two small convolutions add another nonlinear transformation before reducing resolution. It is a useful first change when the team needs more feature composition and wants a regular shape ledger.
But the evidence about small spots adds a condition: do not downsample before the model has extracted their fine detail. The regular block is not an instruction to pool at a fixed schedule. The team should inspect shapes and measure the small-spot slice after each candidate boundary.
Option B: an Inception-like multi-branch stage
input
├─ local branch -----------┐
├─ wider-context branch ---┼─ concatenate -> next stage
└─ pooled branch ----------┘
This choice says that different spatial contexts may matter at the same stage. A small speck and a larger discoloration might need different receptive fields. Inception's central architectural argument was not “parallel branches look clever.” It was improved use of computation while increasing depth and width under a budget, using a multi-scale design.
This option costs more shape bookkeeping and more design choices. It is justified when an ablation or constraint points to competing scales, not merely because the module has a famous name.
Option C: residual blocks for deeper stages
x ───────────────────────┐
│ │
└─ F(x): conv -> act -> conv ── + ──> y
The block output is y = F(x) + x when the shapes match. Plain meaning: retain an identity path and ask the learned branch to add a correction. Technical name: a residual connection or shortcut.
Why does this address a real pressure? The ResNet paper reports that deeper plain networks can be harder to train and reformulates blocks as residual functions relative to their inputs. The shortcut does not prove that every deeper model will improve. It makes a particular information and gradient path explicit, and the paper's evidence was that the residual networks were easier to optimize in its experiments.
For this team, residual blocks become a good experiment if a regular deeper baseline shows a depth-related optimization problem: training error stops improving or worsens when extra plain blocks are added under otherwise controlled settings. If the real failure is loss of small-spot detail after early pooling, residual connections alone are not the first repair.
So far, we have seen that VGG-like blocks, Inception-like branches, and residual blocks answer different questions. This matters because a model change becomes testable: state the bottleneck, make one architectural change, and compare the relevant validation slice and resource cost.
What the Historical Families Actually Teach
LeNet is the small starting model: convolution and downsampling can create a hierarchy whose final stage classifies an image. Its lesson is architectural bias, not that its exact capacity fits every current task.
AlexNet marks a different pressure: scale. Its reported ImageNet system combined a deeper convolutional network with engineering and training choices such as GPUs, ReLUs, data augmentation, and dropout. The correction here is important: an architecture result often belongs to a whole training system. We should not attribute a result to one layer type alone.
VGG makes depth legible through repeated small filters. Inception asks how to distribute computation across paths and receptive-field scales. ResNet makes the training path itself a design concern. Together they replace the vague rule “CNNs get better when they get bigger” with sharper questions:
Do we need another level of feature composition?
Do multiple spatial scales have evidence in this task?
Which operation spends the compute budget?
Does extra plain depth improve optimization or make it worse?
What shape and resolution does the output promise require?
Costs, Limits, and Signals
The trade-off is not old architecture versus new architecture. It is simplicity, compute, resolution, optimization behavior, and evidence quality.
- A regular stack is easy to inspect, but can become heavy as depth and width grow.
- Parallel paths may represent several scales, but they complicate tensor shapes and resource accounting.
- Residual paths can improve the trainability of deeper models, but they do not choose a receptive field, preserve tiny targets, or fix mislabeled data.
Watch the signals that match the suspected boundary. Compare training and validation curves when testing depth. Profile latency and memory when testing branches or width. Break out accuracy by target size and position when choosing downsampling. An overall score alone cannot tell the team whether a residual block fixed optimization or merely changed the failure distribution.
There is also a scope boundary. This lesson does not turn the classic papers into a full architecture search procedure, a hardware-tuning guide, or a survey of modern transformers. It supplies enough vocabulary to inspect a CNN's structural choices and to build the next small model deliberately.
Common Confusions
Confusion: “ResNet means every deeper network will be more accurate.”
Why it is tempting: residual connections made very deep networks trainable in influential experiments. Better model: residual paths address a trainability pressure; data, objective, resolution, regularization, and deployment cost still determine whether added depth is useful.
Confusion: “Inception is just several convolutions running in parallel.”
Why it is tempting: the branch diagram is memorable. Better model: the important design argument is selective multi-scale computation under a resource budget; branch widths and projections are part of that argument.
Confusion: “VGG proves 3 × 3 is always the best kernel size.”
Why it is tempting: the repeated small-filter pattern is easy to reuse. Better model: it was a successful depth-oriented design in a stated setting, not a universal law independent of resolution, task, or compute budget.
Check Your Understanding
Check: A team adds eight plain convolutional layers. Training error becomes worse than in its shallower baseline, while data, optimizer, and evaluation procedure are held fixed. Which classic design move directly tests the suspected pressure?
Think first, then reveal.
Answer: Test a residual block or another shortcut-based design, because the evidence points to a depth-and-optimization problem. Also verify the controlled comparison; this does not show that the data need multi-scale branches or that early downsampling is the cause.
Check: A classifier misses both tiny and broad leaf symptoms, while its compute profile has room for a bounded experiment. Which question makes an Inception-like comparison meaningful?
Answer: Ask whether the task needs useful evidence at multiple spatial scales in the same stage, then compare a resource-accounted multi-branch design with a regular baseline on separate tiny- and broad-symptom slices. A parallel module is not justified only by the presence of two error types.
Transfer: Defend One Architectural Experiment
The team observes two facts: their regular CNN trains well, but misses small spots after its first downsampling stage; a deeper version has the same small-spot failure and increases latency. Choose the first architectural experiment from the following options and defend it:
- add residual blocks everywhere;
- delay or revise the first reduction boundary and compare the small-spot slice;
- replace the model immediately with the newest named architecture.
Model answer: Choose option 2. The direct signal is spatial information loss after the first downsampling stage, and the deeper model did not repair it. Preserve fine resolution longer, or carry an earlier high-resolution feature path, then compare accuracy for small spots alongside latency and memory. Residual blocks target a different pressure—difficulty optimizing depth—and a newer architecture name is not a diagnosis. The experiment may cost more compute, so it needs the stated resource comparison.
Resources
- [PAPER] Gradient-Based Learning Applied to Document Recognition — Focus: the end-to-end convolution-and-subsampling hierarchy in LeNet-5.
- [PAPER] ImageNet Classification with Deep Convolutional Neural Networks — Focus: the AlexNet training system and its large-scale visual-recognition setting.
- [PAPER] Very Deep Convolutional Networks for Large-Scale Image Recognition — Focus: the VGG case for repeated small filters and greater depth.
- [PAPER] Going Deeper with Convolutions — Focus: Inception's multi-scale, compute-aware branch design.
- [PAPER] Deep Residual Learning for Image Recognition — Focus: residual functions, identity shortcuts, and the reported optimization result.
Key Takeaways
- Read a classic CNN as a response to a named bottleneck, not as a historical name to memorize.
- VGG-like regular depth, Inception-like multi-scale branching, and residual shortcuts make different trade-offs and require different evidence.
- A structural change is useful when a controlled validation slice and resource signal support its pressure; a famous architecture name is not that evidence.
- Classic CNNs prepare you to read and implement a bounded model, while task-specific data and constraints still decide the final design.