Domain Adaptation and Few-Shot Learning

By the end of this lesson, you will be able to...

• distinguish a changed target distribution from a shortage of target labels, even when both appear together;

• design a bounded adaptation experiment with a target baseline, a shift diagnosis, and an uncertainty-aware evaluation plan;

• choose when to preserve a representation, gather evidence, use a support-set comparison, or attempt carefully scoped adaptation.

Idea in one sentence: Domain adaptation asks whether the world changed; few-shot learning asks what can be justified with very few labels; a good plan measures those pressures separately before it changes the model.

Core Insight

Suppose the warehouse team moves its defect model to a second facility. The task names are unchanged: classify a package photo as damaged or not damaged. But the new facility has a dimmer camera, darker packaging, and a different belt background. It supplies 3 reviewed examples of each label—12 images in total—and thousands of unlabeled photos from the new camera.

The tempting response is “fine-tune the old model on the 12 labels.” The team has a pretrained representation, and the previous lesson showed how to unfreeze a late block carefully. It is a reasonable tool, but it is not yet a diagnosis. Twelve labels can be too little evidence for a broad update; the new lighting can create a distribution shift that a head-only model also cannot handle.

There are two pressures:

domain shift: the target inputs differ from the inputs that trained the source model
few-shot setting: the target labels are too scarce to estimate a new decision reliably by ordinary supervised training

They may happen at once, as in this facility move, but they are not the same failure. Domain adaptation is about handling a relevant source-target distribution difference. Few-shot learning is about making a decision from a very small labeled support set, usually by relying on prior representation structure rather than relearning everything.

The Promise We Need to Keep

The product promise is narrow: route a new-facility photo to an inspector without claiming a level of certainty that the 12 labels cannot support. The model should not silently treat “unlabeled target photo” as “evidence of a known label,” nor should it allow a convenient embedding neighbor to substitute for a reviewed defect decision.

The team needs to answer four different questions:

The first two questions concern representation and shift. The third concerns label scarcity. The fourth turns an uncertain model result into an operational boundary instead of a hidden risk.

The Naive Design: Treat Every New Facility as Ordinary Fine-Tuning

Ordinary fine-tuning assumes that the target labels are sufficient to guide model updates and validate their result. That can be a good design when the target data are diverse and representative enough for the decision at stake.

It breaks here for two separate reasons. First, a 12-image support set may not cover the facility's camera angles, package materials, damage severities, or backgrounds. A model can fit those examples while learning a brittle coincidence. Second, the source and target feature distributions may differ even for the same labels: glare, sensor response, and background can move target images in the embedding space from the previous lesson.

Unlabeled target data can establish that target inputs differ from source inputs or expose a cluster by camera condition. It cannot establish that a target cluster has a particular defect label. Labels and input distributions answer different questions. This distinction is central to domain-adaptation methods such as domain-adversarial training, which explicitly use labeled source data and unlabeled target data to learn features less able to distinguish the domains. That is one approach, not a guarantee that every shift should be aligned. Ganin et al. provides a technical example.

A Better Boundary: Diagnose Before Choosing an Intervention

The stronger design treats the facility move as a small decision matrix. The observations below are illustrative, not measured warehouse results.

This table prevents an attractive but wrong shortcut: “there are few labels, so we need few-shot learning” or “the camera changed, so we need domain adaptation.” The needed intervention follows the dominant evidence and the decision risk.

Work a Bounded Experiment

Start with a fixed checkpoint and the embedding contract from lesson 027: encoder version, selected layer, preprocessing, vector preparation, and metric. Split the reviewed target cases by facility condition when the data permit it. Do not tune repeatedly on the same 12 photographs and then call their final score validation.

Here is a compact experiment plan for the new facility:

Candidate B is a few-shot-style use of a strong prior representation: a small labeled support set helps define the local decision without claiming that the model learned a new task from scratch. Matching Networks is a classic formulation that maps a labeled support set and an unlabeled example into a label decision through learned similarity. It illustrates a design family, not a reason to deploy nearest neighbors without local evaluation. Vinyals et al. is the primary reference.

Now trace a useful decision path. The team finds that the frozen reference misses dim-light damage cases. In the embedding audit, dim-light images cluster together regardless of label. The 12 labels contain only one dim-light damaged package and no dim-light intact package. This is synthetic evidence for the lesson.

target photos cluster by dim lighting
        +
support examples do not cover both labels under dim lighting
        ->
we have evidence of an input shift and evidence of insufficient label coverage
        ->
do not infer a new defect boundary from the one dim-light example
        ->
collect reviewed examples across the missing condition; meanwhile route that slice to inspection

The initial model, “fine-tune now,” would turn a missing-evidence problem into parameter movement. The better model makes the gap visible and assigns a safe response. If later the team gathers independent dim-light examples for both labels, it can compare candidates C and D against the frozen reference with the staged controls from lesson 026.

So far, we have seen that a changed feature geometry and a tiny support set create different uncertainties. This matters because the team can preserve useful transfer while refusing to learn a broad conclusion from one narrow example.

The Trade-off: Adaptation Versus Evidence

Adaptation can improve fit to a new facility. It costs labels, experiment time, version management, and the possibility of overfitting or losing useful source behavior. A support-set or frozen-feature approach can be fast and reversible. It costs coverage: its decision is only as good as the representation and the reviewed cases near the query.

Collecting more target labels can be slower than changing a model, but it may be the only action that resolves label uncertainty. Using unlabeled target data can reveal shift and help formulate representation hypotheses, but it cannot validate class-specific quality by itself. Escalating uncertain slices to humans costs operational capacity but protects the promise while evidence is thin.

The choice is therefore situated. Conservative preservation and human review are preferable when error cost is high, the new condition is underrepresented, and target labels are sparse. Bounded fine-tuning becomes more reasonable when diverse target labels, an independent evaluation set, and a specific mismatch justify it. No fixed number of “shots” or target images makes that decision universally safe.

Review Check: Connect the Transfer Cluster

This lesson closes the transfer-and-adaptation cluster. Use the warehouse move to connect its four ideas:

Check Your Understanding

Check: A facility's unlabeled target images form a distinct embedding cluster by camera. What does that observation justify, and what does it not justify?

Think first, then reveal.

Answer: It justifies investigating a camera-linked distribution shift and checking frozen-model behavior by that slice. It does not identify the cluster's labels, prove target quality is poor, or prove that a particular domain-adaptation method will help.

Check: A team has three target examples per label and gets 100% accuracy after full-model fine-tuning on those same six examples. What has it demonstrated?

Think first, then reveal.

Answer: It has demonstrated fit to those six examples, not reliable target generalization. The score is not independent validation, and the support set may not cover the target conditions that matter.

Practice: Design the New-Facility Gate

A third facility changes both camera angle and package material. You receive many unlabeled photos and two reviewed examples per label, none from the night shift. Design a first-release gate. State:

• one shift hypothesis and one label-coverage gap;
• two comparisons or audits to run before adaptation;
• a condition that would justify collecting labels rather than fine-tuning; and
• a safe handling rule for night-shift photos until the evidence improves.

Model answer: The shift hypothesis is that camera angle and material move images in the representation space; the coverage gap is the absence of night-shift examples for either label. Audit embedding neighborhoods and frozen-model errors by facility condition, then compare a fixed support-set method with the frozen reference only on independently reviewed cases when available. Collect labels if the target slice is missing one or both classes, if neighbors are dominated by camera/material rather than defect, or if the small support set cannot produce a stable evaluation. Until then, route night-shift photos to inspection or abstain according to the product policy; do not treat their lack of labels as a reason for confident automatic classification.

Resources

• [PAPER] Domain-Adversarial Training of Neural Networks — Focus: a source-labeled, target-unlabeled approach to domain-invariant features.
• [PAPER] Matching Networks for One Shot Learning — Focus: using learned similarity with a small labeled support set.
• [BOOK] Dive into Deep Learning: Fine-Tuning — Focus: controlled adaptation choices after a transfer baseline.

Key Takeaways

• Domain adaptation is about a relevant source-target distribution difference; few-shot learning is about severe target-label scarcity. They often co-occur but require different evidence.
• Unlabeled target data can reveal shift, while reviewed target labels are needed to support class-specific quality claims.
• Start with a fixed representation and a bounded comparison; do not use training fit on a tiny support set as proof of adaptation.
• When a target condition lacks coverage, collecting labels or routing to review can be more honest and useful than forcing a broad model update.