Cloud Deployment

By the end of this capstone, you will be able to...

• assemble a single-model inference delivery dossier that connects model evidence, a resource budget, an API contract, and a reproducible image;

• decide whether a bounded release is ready, blocked, or must be rolled back from evidence rather than from a successful build alone;

• mark the ownership boundary between a reviewable inference artifact and the platform, monitoring, and fleet work owned by a continuation track.

Idea in one sentence: A deployable neural inference system is a traceable promise—one approved pipeline, one resource envelope, one client contract, one runtime image, and named evidence for both advancing and stopping the release.

The Scenario

The warehouse team has reached the end of its defect-classification work. Its pipeline is named defect-2026-03: image decoding, resizing and normalization, a smaller neural backbone, a review threshold, and a label mapping. Lessons 029–031 produced three useful but separate artifacts:

• a constraint sheet says the target device needs p95 end-to-end latency of at most 60 ms, peak memory of at most 1.0 GB, and no more than a one-point quality loss on the reviewed target set or dark-package slice;
• an API contract says one JPEG request returns a versioned decision, or a named validation or availability failure;
• an image contract pins the runtime and pipeline artifact, starts as a non-root user, and passes an image-level fixture request.

The team now wants to send defect-2026-03 to a small set of scanners. Someone says, “The Docker image works, so deploy it to the cloud.” That is not a release decision. It leaves unanswered which exact evidence belongs to the version, what the client can rely on, what condition stops expansion, and who owns work beyond this one inference artifact.

This capstone does not ask you to select a cloud provider, configure a cluster, or design fleet autoscaling. Those are valid production concerns, but they belong to ml-systems-and-training-infrastructure. Here, the deliverable is smaller and more durable: a neural inference delivery dossier that lets another team receive one bounded release without guessing.

Constraints

Treat the following values as illustrative teaching data, not universal thresholds. They define the promise for this scenario.

The dossier must preserve the measurement boundary. “Latency: 56 ms” is ambiguous. “p95 from accepted JPEG request to versioned response, on the stated target device during the representative ten-minute belt workload” is a claim someone can test again.

Core Insight

One Inspectable Delivery Claim

The initial model is a folder of individually plausible facts: a checkpoint, a benchmark screenshot, an endpoint, and an image. It fails because none proves that the others describe the same pipeline. A client could call a new API against old preprocessing; a fast benchmark could omit decoding; an image could load a mutable artifact. Each fact might be true while the release claim is false.

The stronger model is a dossier with one release identity at its center:

pipeline manifest + image digest
          |
          +--> quality and slice evidence
          +--> target-device resource evidence
          +--> API fixture evidence
          +--> image startup evidence
          +--> advance / hold / rollback rules

The arrows do not mean that an image digest proves model quality. They mean every evidence item names the same pipeline version and can therefore support a bounded decision about it. The learner's job is to expose missing links, not to make an impressive-looking deployment diagram.

The Proposed Delivery Dossier

1. Model card and pipeline identity

Start with the smallest record that tells a reviewer what is being released and what it is not known to do.

The digest is illustrative. In a real dossier it must be the immutable digest produced by the reviewed build, not a tag such as latest. Notice also that the model card does not promise capacity, regional availability, or incident response. Those are different system claims with different owners.

2. Resource envelope and evidence

Attach the final candidate to the exact budget from lesson 029. The example below is synthetic, but it demonstrates a decision record rather than a vague performance assertion.

Each row requires its test context: target device, pipeline manifest, representative image mix, load duration, and timer boundary. Without that context, a later reviewer cannot tell whether a faster result used a different input shape or skipped image decoding. A pass is not a permanent property of a model file; it is evidence under a stated contract.

3. API and failure contract

The delivery dossier must name the client promise from lesson 030. It need not copy every schema field, but it must show that the runtime test exercises the declared behavior.

{
  "request_id": "scan-7f32",
  "pipeline_version": "defect-2026-03",
  "decision": "review",
  "label": "damaged",
  "score": 0.52,
  "score_semantics": "model confidence before the review threshold",
  "latency_ms": 48
}

The following table keeps failures honest.

A review result is not a service failure. The pipeline completed and the product policy declined to automate that case. This distinction is part of the neural system's boundary: it prevents uncertainty from being silently converted into a confident label or a misleading server error.

4. Image and startup contract

The image is the execution unit, not the whole release. Record enough to prove that its runtime matches the pipeline and API evidence.

For this capstone, the image-level fixture is a release gate. It does not replace target-device load testing or quality evaluation. It connects the tested API surface to the packaged runtime, while the resource sheet and model card retain their separate kinds of evidence.

Walkthrough: Decide Whether to Advance

The team proposes a deliberately small rollout sequence. The percentages describe a release decision, not an autoscaling design.

The operational measurements at stages 1 and 2 should be designed with the platform team. This lesson does not prescribe their telemetry stack, traffic routing, replica policy, or incident process. It does require the release owner to state what evidence would make continued exposure unsafe and which approved pipeline is the rollback target.

For example, suppose a limited-use fixture shows that the response emits pipeline_version: defect-2026-02 even though the image is labeled for defect-2026-03. The correct decision is hold, not “the endpoint is up.” The identity chain is broken: either the image did not load its declared artifact or the API response is not attached to it. The team must restore the matching artifact and rerun the dossier gates before sending more traffic.

Now suppose the candidate returns defect-2026-03, stays within the p95 budget, but reflective-package cases are being sent straight to damaged rather than review. The correct decision is also hold or rollback. The failure is not necessarily network availability; it is a mismatch between the reviewed threshold policy and the client-visible behavior. The safer previous pipeline is the named rollback target until the policy and fixtures agree again.

So far, we have seen that a rollout is evidence-based when it can stop for an identity, contract, resource, or protected-quality failure. This matters because a green build and an available endpoint are only two narrow facts, not proof that the neural inference promise is being kept.

Failure Review: Find the Missing Owner

Use this review table when a proposed dossier feels complete but a claim has no evidence or owner.

The handoff is not a disclaimer. It is a quality feature. A later ML-systems team can use the dossier's manifest, image digest, API contract, load profile, and rollback target as input. They should not have to infer them from a notebook, an unversioned checkpoint, or a deployment screenshot.

Trade-offs and Limits

A tighter dossier costs time: fixture maintenance, frozen evaluation data, a representative benchmark, and a named release owner. In return, it reduces the risk that different teams reason about different artifacts while using the same model name.

It also constrains claims deliberately. The dossier can support “this pipeline met these checks on this device and workload.” It cannot support “the service is reliable everywhere,” “the score is a calibrated probability,” or “the platform can absorb any traffic.” Expanding those claims requires the surrounding systems evidence that this track explicitly defers.

There is a real tension between release speed and evidence. A mutable model tag can make a quick experiment easy, but it makes a rollback and audit ambiguous. A large baked-in artifact can make image identity strong, but it may increase transfer and startup cost. A review band can reduce unsafe automated decisions, but it creates human-workload pressure. The correct choice follows the named product constraints; no single deployment substrate or neural optimization is automatically best.

Final Challenge: Produce the Dossier

You are given a candidate pipeline with these facts:

• defect-2026-04 has p95 latency of 54 ms and peak memory of 0.8 GB on the stated target workload.
• Its overall target-set quality is 94.2%, but its reflective-package slice is 88.7% against a required floor of 89.5%.
• The container starts and returns pipeline_version: defect-2026-04 for the happy-path fixture.
• The API returns 400 invalid_request for a missing image, but returns 200 damaged when the runtime times out.

Write a one-page release decision with:

1. the release identity and intended-use boundary;
2. a compact evidence table that separates passes from failures;
3. the API correction required before any rollout;
4. an advance, hold, or rollback decision and the named rollback target; and
5. two responsibilities you hand off to ml-systems-and-training-infrastructure.

Rubric

Model answer: Record defect-2026-04 as the candidate pipeline and state that it classifies only supported package images under its declared preprocessing and threshold policy. The resource measurements pass, as does the happy-path image identity fixture. The candidate nevertheless fails approval because the reflective-package slice is 88.7%, below the protected 89.5% floor. It also fails the API contract: a timeout means inference did not complete, so returning 200 damaged fabricates a model decision. Change that path to the declared 503 inference_unavailable response and add a failure-injection fixture. The decision is hold defect-2026-04; if it has received any limited traffic, return it to the last approved pipeline, defect-2026-03, until the slice and timeout evidence pass. Hand off rollout topology and capacity policy, plus ongoing monitoring and incident response, to the ML-systems track. No latency result justifies releasing a pipeline whose protected quality and failure semantics are false.

Resources

• [DOCS] Dockerfile reference — Focus: making the build and runtime inputs of an image inspectable.
• [DOCS] RFC 9457: Problem Details for HTTP APIs — Focus: expressing client-readable failure categories without converting an unavailable inference into a false prediction.
• [BOOK] Designing Machine Learning Systems — Focus: connecting model, data, and serving choices to explicit operational evidence.

Key Takeaways

• A deployment-ready neural artifact is a dossier of linked evidence, not a checkpoint, endpoint, or container considered alone.
• Every release claim must name the same complete pipeline and preserve its measurement boundary, protected slices, and client contract.
• Advance only when identity, quality, resource, API, and image gates all pass; hold or roll back when any required claim fails.
• The dossier makes an explicit handoff possible: it does not substitute for orchestration, fleet capacity, monitoring, or incident operations.