Containerization with Docker

By the end of this lesson, you will be able to...

• describe a container image as a versioned runtime contract for an inference pipeline, not merely a way to ship Python code;

• trace how a Docker build assembles a base runtime, dependencies, application, artifact reference, and startup command;

• review an ML-service image for reproducibility, size, secret handling, and the boundary between packaging and deployment operations.

Idea in one sentence: A container image makes an inference runtime reproducible only when its dependencies, artifact assumptions, configuration boundary, and startup behavior are explicit and testable.

Core Insight

Suppose the warehouse team has a working inference API from the previous lesson. It loads pipeline defect-2026-03, validates a JPEG, applies the approved preprocessing, and returns a versioned response. On the engineer's laptop, it works. In staging, it fails to decode images because a native image library is missing. A later build succeeds but loads a newer package version that changes a default behavior.

The tempting explanation is “the server works; staging is different.” That sentence names the symptom, not the missing boundary. The laptop contains undeclared state: an operating system, system libraries, Python versions, installed packages, caches, and a startup environment. The inference pipeline cannot be reproduced by copying only app.py and a checkpoint.

Containerization packages the runtime assumptions into a build artifact called an image. A running container is an instance of that image. The image should make it possible to answer: what code, libraries, pipeline artifact, configuration interface, and command created this behavior?

The Runtime We Need to Reproduce

An ML inference service has more than model weights. For the warehouse API, its runtime contract includes:

Secrets do not belong in an image layer or Dockerfile. An image can declare that it expects a credential at runtime, but a deployment environment should provide the sensitive value through its approved mechanism. Likewise, a container filesystem is suitable for the packaged application and temporary work; it is not a substitute for durable business state.

The Initial Model: docker build Makes It Reproducible

Building an image is an important step, but it does not automatically make an inference service reproducible. A Dockerfile that installs unpinned dependencies, copies a local model with an unclear origin, and starts a server through a shell profile can produce an image while leaving its behavior ambiguous.

The initial model works for an exploration: an image may be enough to prove that a program can start somewhere other than the laptop. It breaks when the image must be rebuilt, reviewed, or compared with a later pipeline version. Reproducibility needs declared inputs and a verifiable output, not simply a successful build command.

The stronger model is a build contract. It states the intended base, dependency inputs, copied source, model-artifact decision, runtime user, and startup command. It also preserves a test that exercises the actual image, not just source code in a local virtual environment.

Build the Smallest Honest Image

Here is a conceptual Dockerfile for the inference API. Names and versions are illustrative; the structure is the teaching point.

FROM python:3.12-slim

WORKDIR /service
COPY requirements.lock ./
RUN pip install --no-cache-dir -r requirements.lock

COPY app/ ./app/
COPY artifacts/defect-2026-03/ ./artifacts/defect-2026-03/

ENV PIPELINE_VERSION=defect-2026-03
USER 10001
CMD ["python", "-m", "app.server"]

Read it as a sequence of state changes:

This is a teaching model, not a universal production Dockerfile. A GPU runtime may require a compatible base and a host runtime that grants device access. A model artifact may be baked into the image for a tightly coupled, reproducible release, or fetched by a verified version at startup when image size and update cadence make that preferable. Both choices have costs.

A Worked Image Audit

The team compares two hypothetical builds. The measurements and names are illustrative.

Image A may run today. It cannot prove what it will run after a dependency update, whether a secret was copied into a layer, or whether a new machine has the same model cache. Image B gives a reviewer an inspectable chain from declared inputs to runtime behavior.

The useful verification is an image-level test:

build image with its declared inputs
  -> start that image with non-secret test configuration
  -> send a supported fixture image to the API
  -> assert the response pipeline_version, schema, and expected test decision
  -> stop the container

This test does not prove real-world model quality. It proves a narrower but vital claim: the packaged runtime loads the intended pipeline and executes the serving contract. The quality and performance evidence from lessons 029 and 030 remains separate and must be attached to the same artifact version.

So far, we have seen that an image is trustworthy when its build inputs and runtime behavior can be inspected together. This matters because “works in Docker” becomes a specific claim that can be tested, rather than a vague substitute for reproducibility.

Trade-offs: Bake, Fetch, or Leave Outside

The artifact decision is the central ML-specific trade-off.

Bake the model artifact into the image when a single image should carry a tightly coupled, reviewed pipeline. This improves portability and makes the image digest a strong release identity. It costs image size, registry transfer time, and a new image build for artifact changes.

Fetch a versioned artifact at startup when the artifact is large or changes on a cadence different from the service code. This can reduce image movement, but adds startup time and a runtime dependency. The service must verify the artifact identity before readiness, not silently take “latest.”

Mount or inject an artifact may fit a controlled environment, but only if the source, version, permissions, and readiness behavior remain explicit. A mutable host path is not a reproducibility strategy.

No option solves orchestration, rollout policy, autoscaling, or host-level security. Docker packages the execution unit. The next capstone will decide how that unit, its API, its model evidence, and its delivery checks fit together; platform operation remains a handoff to the ML-systems track.

Common Failures and Signals

Symptom: the service works locally but fails in the image.

Likely cause: a local package, system library, environment setting, or file path was undeclared.

Signal and response: compare the image's declared build inputs with the failing dependency; add the required dependency or remove the hidden assumption, then rerun the image-level fixture test.

Symptom: a rebuild changes predictions without a code change.

Likely cause: a mutable base image, dependency range, or unversioned artifact changed the runtime.

Signal and response: record immutable build inputs and pipeline identity; use the resulting image digest and the pipeline version as release evidence.

Symptom: the image is small but cannot serve after start.

Likely cause: a required artifact was omitted, a startup fetch failed, or readiness was declared before loading completed.

Signal and response: test the actual startup path and make readiness depend on the declared inference pipeline being available.

Check Your Understanding

Check: A Docker image contains API code and pinned Python packages, but downloads an unversioned latest model at startup. Is the complete inference pipeline reproducible?

Think first, then reveal.

Answer: No. The code runtime is more reproducible, but the model input is mutable. The service needs a versioned or content-verified artifact reference and readiness behavior that confirms it loaded that intended artifact.

Check: Why is an image build that succeeds weaker evidence than a test that starts the image and calls its API with a fixture image?

Think first, then reveal.

Answer: A build proves the recipe assembled. The image-level request proves that the assembled runtime can start, load its declared pipeline, and execute the client-facing inference contract. It still does not replace quality or load testing.

Practice: Review an Image Contract

An image uses python:latest, installs requirements.txt with open version ranges, copies an API key into the build, and downloads a model named current at startup. Write the smallest revision plan that makes the image suitable for a reviewable inference release.

Model answer: Choose a declared compatible base image, resolve and lock dependencies, and copy the lock file before application source for stable caching. Remove the API key from all image layers and inject it at runtime through the deployment's secret mechanism. Replace current with a versioned or content-verified model reference; make readiness wait until that artifact and the declared preprocessing pipeline load. Record the pipeline version and image digest, then start the image in a test and send a fixture request that verifies the response contract. This does not add autoscaling or rollout policy; those are outside the image boundary.

Resources

• [DOCS] Dockerfile Reference — Focus: how build instructions define image layers and runtime defaults.
• [DOCS] Docker Build Cache — Focus: ordering stable and changing build inputs for fast, explainable rebuilds.
• [DOCS] Docker Build Checks — Focus: automated checks for common Dockerfile issues.

Key Takeaways

• A container image packages the full inference runtime contract: base, dependencies, application, artifact assumption, configuration interface, and startup behavior.
• A successful build is not enough; start the image and exercise a versioned fixture request to prove the packaged pipeline works.
• Artifact placement is a trade-off among reproducibility, image size, startup dependency, and release cadence; never rely on mutable “latest” state.
• Docker creates a reproducible execution unit, not autoscaling, rollout, secrets management, or fleet operations.