Capstone: Build a World Model Dossier

By the end of this lesson, you will be able to...

• Build a small world model dossier with a purpose, boundary, variables, proxies, feedback, uncertainty, and evidence.

• Compare an initial model with an alternative and state what each one can and cannot explain.

• Define update triggers, a review date, and a rubric for deciding whether the model should be repaired, narrowed, or retired.

Idea in one sentence: A world model dossier is a compact, inspectable argument about a situation, including the evidence and conditions that could change it.

Core Insight

The track began with a useful warning: a model is a purposeful simplification, not reality in a smaller box. The capstone turns that warning into an artifact another person can inspect.

Imagine an engineering team that has missed three delivery dates. A manager proposes adding two engineers. A product lead proposes freezing scope. A staff engineer points to review queues and rework. Each person has a plausible model. None is yet a dossier.

A dossier makes the disagreement concrete:

What decision are we making?
What part of the situation are we modeling?
Which variables and signals support the claim?
What is uncertain or hidden?
Which alternative model would change the decision?
What observation would make us revise or retire this model?

The goal is not to produce a perfect theory of delivery. The goal is to produce a small model that improves one decision and leaves its assumptions visible.

The Scenario

The team owns a service used by an internal operations group. It planned 12 work items for a six-week cycle. Seven shipped by the target date. The other five were delayed, and two of the seven required follow-up fixes.

The team has these observations:

The decision is deliberately narrow:

For the next six-week cycle, should the team add capacity, reduce committed scope, or change the review and rework process?

This is enough pressure for a useful model. It is not enough evidence for certainty.

Constraints

Every dossier needs constraints. Without them, a model can quietly expand until it explains nothing.

For this scenario, use these constraints:

• The decision concerns one team and one six-week cycle.
• The team cannot change the product roadmap outside that cycle.
• The dossier must use signals the team can collect weekly.
• It should recommend at most two interventions.
• It must state what remains outside the model, including individual motivation and larger company politics.

Constraints are not an inconvenience added after the “real” reasoning. They define what the model is allowed to claim.

Design Goal

Write the goal as a decision and an outcome, not as a wish to understand everything.

Decision: choose the smallest intervention for the next six-week cycle.
Desired outcome: increase on-time delivery without increasing harmful rework.
Success signals: on-time rate, review wait, unplanned-work share, and escaped fixes.
Review point: the end of the next cycle, with a weekly check for early drift.

The phrase “without increasing harmful rework” matters. A team can ship more items by skipping tests, then pay for that speed later. The model must keep the trade-off visible.

Proposed Model

Build the dossier in layers. Each layer should answer one question.

Purpose and boundary

Purpose: choose an intervention that improves reliable delivery in one cycle.
Inside: planned work, review flow, unplanned work, rework, and staffing.
Outside: market demand, individual motives, company politics, and long-term architecture.

The boundary does not say that outside forces are unimportant. It says they will not be modeled in detail for this decision.

Variables, signals, and proxies

The model does not pretend that a signal is the outcome itself. It records how the signal can mislead us.

Feedback and delayed consequences

The initial causal sketch is:

More committed work
  -> more parallel work
  -> longer review wait
  -> more context switching
  -> more defects and rework
  -> less capacity for planned work
  -> more pressure to commit too much

There is a delay between skipping review and seeing a production fix. That delay can make a bad intervention look successful during the first week.

Uncertainty and alternatives

Use ranges and competing models instead of one confident story.

Current on-time rate: about 50-70%, depending on how “on time” is counted.
Review-wait contribution: likely material, but not yet separated from item size.
Unplanned-work share: roughly 15-25% over recent cycles.

Keep three small models:

The portfolio is disciplined because every model has a question, signal, decision, and blind spot.

Walkthrough: From Dossier to Decision

Start with the naive recommendation:

Add two engineers. More people create more delivery capacity.

Now trace what the dossier predicts.

The alternative intervention is narrower:

Commit 10 items instead of 12, limit work in progress,
assign a daily review owner, and reserve 15% capacity for fixes.

That intervention makes a different prediction:

If review wait is a main constraint, median review wait should fall below two days,
on-time delivery should rise, and escaped fixes should not rise after the cycle.

The model is now useful even if the prediction fails. A failure would tell us whether review wait was not the main constraint, whether the signal was poorly measured, or whether another model deserves more weight.

Evidence Review Meeting

Treat the dossier as a living handoff, not a document that is written once and filed away. The team reviews the same claims at three points.

The dossier should record these updates rather than silently overwriting its first version. A short log might look like this:

Initial claim: review wait is a major constraint.
Week 1: plausible, but evidence is thin; keep the test running.
Week 3: prediction is holding; continue the flow intervention.
Week 6: on-time rate improved without a large quality penalty.
Next trigger: if review wait rises above three days for two weeks,
or escaped fixes exceed two items, reopen the model.

This log makes uncertainty operational. The team does not need to pretend that the first explanation was complete. It needs to show how observations changed confidence and action.

What the Dossier Is Not

The capstone artifact is not a project plan, a dashboard export, or a long explanation of every possible cause. It is also not a verdict about who is responsible for a failure.

A project plan tells people what to do. A dashboard displays selected signals. A dossier explains why selected signals support a bounded decision, where they can mislead, and what would cause an update. The same team may need all three, but they answer different questions.

Keep the artifact small enough to review. If a reader cannot locate the purpose, boundary, prediction, and update trigger in a few minutes, remove detail or move it to supporting evidence. The model should be compact; the evidence behind it can be larger.

Failure Review

Before accepting the dossier, try to break it.

Boundary challenge

What if a major dependency outside the team blocks two items? The dossier should mark those items as dependency-blocked instead of calling every delay a team-flow problem. If dependency failures are common, the boundary must be widened or the decision narrowed.

Proxy challenge

What if “on-time delivery” rises because the team splits large items into tiny tickets? Track item size, outcome completion, and follow-up fixes. A better-looking proxy can still hide a worse result.

Feedback challenge

What if the rework signal appears only after the cycle ends? Keep the weekly review, but do not declare success until the delayed signal arrives. A delayed consequence is still part of the model.

Assumption challenge

What if the team has no control over review ownership? Then the proposed intervention is not actionable. Replace it with a decision the team can actually make, or state the dependency as an explicit constraint.

Alternative-model challenge

What if the flow model predicts improvement but the scope-and-incentives model predicts repeated overcommitment? Do not average the models into a vague score. Run the smallest test that distinguishes them, such as comparing committed scope with completed outcomes for one cycle.

Trade-offs

A dossier improves judgment by making a model inspectable. It also creates work.

It helps when:

• several people need to reason about the same decision;
• evidence is incomplete but action cannot wait forever;
• the situation can be checked again after an intervention.

It costs:

• time to define signals and collect evidence;
• the discomfort of naming uncertainty and excluded forces;
• maintenance when the situation or decision changes.

It does not guarantee:

• that the chosen intervention will work;
• that every important variable is observable;
• that values and politics can be reduced to measurements;
• that a model built for one cycle applies to another context.

The boundary is visible when the dossier becomes a polished report no one uses, when its signals arrive too late for the decision, or when every failure produces an untestable new explanation.

Evidence and Readiness

Use this readiness rubric before sharing the dossier.

Readiness does not mean “true.” It means another person can inspect the claim, challenge it, and help update it.

Final Challenge: Build Your Dossier

Choose a bounded question from work, study, health, a product, or a personal project. Avoid a question so broad that it has no decision owner. Then produce a one- to two-page dossier with these headings:

1. Decision and purpose — What choice will this model support?
2. Boundary — What is inside, outside, and temporarily ignored?
3. Variables and signals — What matters, what can you observe, and what is only a proxy?
4. Mechanism — What changes what? Include a feedback loop, delay, or state transition when relevant.
5. Evidence and uncertainty — What do you know, how strongly, and what is missing?
6. Alternative models — What other explanation could guide a different action?
7. Prediction and update trigger — What should happen, by when, and what would force a revision?
8. Trade-offs and limits — What becomes more expensive, and what remains unsolved?

Your dossier is ready when a skeptical reader can answer three questions without asking you to fill in the blanks:

What decision does this model support?
What observation could change it?
What should we do if that observation appears?

What Comes Next

This track stops at the point where a human model becomes explicit, revisable, and useful for a bounded decision. The next tracks deepen different parts of the practice:

• epistemology-and-knowledge-systems examines belief, testimony, disagreement, and trust.
• decision-making-uncertainty-and-judgment connects models to choices under stakes and values.
• scientific-reasoning-and-philosophy-of-science studies measurement, causality, replication, and theory change.
• system-dynamics-and-causal-modeling formalizes feedback, stocks, flows, and simulation.

The dossier is a handoff artifact. It lets those deeper tracks challenge a model without first having to guess what the model was.

Resources

• [BOOK] The Model Thinker — Focus on using several models and matching each one to a question.
• [BOOK] Thinking in Systems — Focus on boundaries, feedback, delays, and unintended consequences.
• [ARTICLE] Mental Models I Find Repeatedly Useful — Focus on turning a mental model into an inspectable decision aid.

Key Takeaways

• A dossier starts with a decision and a boundary, not a list of interesting concepts.
• Variables, signals, and proxies must be separated so a convenient metric is not mistaken for the outcome.
• Feedback, delays, uncertainty, and competing models make the reasoning more honest and more useful.
• A strong model includes a prediction, a review date, and a disconfirming signal before action is taken.
• Readiness means inspectable and revisable—not certain, universal, or permanently correct.