Model Failure and Repair

By the end of this lesson, you will be able to...

• Diagnose a model failure by separating prediction misses, boundary drift, stale assumptions, missing variables, and misleading proxies.

• Repair a model with a clear update, test, and retirement rule instead of defending it with patches.

• Review a model change by stating what improves, what becomes more expensive, and what evidence would still disconfirm it.

Idea in one sentence: A model is repaired when its assumptions, boundaries, variables, and predictions are changed in response to evidence—and retired when another model is more useful.

Core Insight

Imagine a support team deciding whether to move one engineer from feature work to the ticket queue. Its simple model says:

10 new tickets per day
12 tickets resolved per day
therefore the backlog should fall by 2 per day

The manager makes the move. Five days later, the backlog is 47 instead of the predicted 34. The dashboard says the team is closing more tickets. Customers still report long waits.

The first reaction is often emotional: the model was wrong, so someone must be wrong. Ask:

Which part of the model failed, and what should the next version do differently?

The model may have counted arrivals badly. It may have ignored a release that created a new ticket class. It may have used “tickets closed” as a proxy for service quality while old complex tickets stayed open. Or its boundary may have stopped at the support queue even though product changes were creating demand.

Model failure happens whenever a simplified representation guides a decision and reality pushes back. Repair means comparing prediction with observation, locating the mismatch, changing what evidence supports, and testing the new version.

The Review Map

The previous lessons built a set of tools. This lesson connects them into a failure review.

This prevents discarding a useful model too quickly or protecting it with purposeless patches.

The Small Situation

Write down the original model before changing it. That makes the claim inspectable.

Purpose: decide whether one extra engineer will reduce customer wait time.
Boundary: the support queue only.
Variables: tickets arriving, tickets resolved, open backlog.
Proxy: tickets closed per day represents service improvement.
Prediction: 44 open tickets becomes about 34 after five days.
Uncertainty: +/- 3 tickets, assuming normal demand.

Now compare the five-day observation:

The simple model was not useless. It correctly showed that resolution capacity matters. It was incomplete for the decision because it treated demand as stable and treated closure count as service quality.

Where the Naive Repair Breaks

The naive repair is to change a number until the old model fits:

Resolution capacity was not 12. It was 13.5.
Let us change the forecast and move on.

This is tempting because it is fast. It also hides the cause. If a release created 18 extra tickets, a more precise capacity number will not prevent the next release from doing the same thing.

A second naive repair is to add every possible cause:

Include staffing, releases, user sentiment, documentation, holidays,
team mood, browser versions, and weather.

That turns the model into a catalogue. Each variable needs a reason, an observable signal, and a decision it can change.

The target is not a model that explains everything. It should make the next decision less surprising.

Five Ways a Model Can Fail

Prediction miss

The model predicted one result and the observed result fell outside the stated range.

In the support case, 47 is far from the expected 34 +/- 3. First check the data and timing.

Boundary drift

The world changed at the edge of the model, or the decision quietly expanded beyond the original purpose.

The model was about queue capacity. The decision became “how do we reduce customer wait after a release?” Product changes now belong in the frame.

Stale assumption

An assumption that was once reasonable stopped matching current conditions.

“Demand is about 10 tickets per day” may have been true last month, but not after the release.

Missing variable or mechanism

An important cause was outside the variable list. A new ticket category, a feedback delay, or a re-opened ticket can change the result without violating the variables already present.

Misleading proxy

The measured signal moves, but the thing we care about does not. Closure count rose while wait time worsened because easy tickets were closed first.

These failures can occur together. Find the smallest combination that explains the miss and changes a decision.

A Worked Repair Trace

The team investigates the five-day miss in stages.

The repaired portfolio now has explicit jobs:

Capacity model:
  Question: Can current staff process the current arrival rate?
  Signal: arrivals, resolution rate, age distribution.

Release-cause model:
  Question: Did a change create preventable demand?
  Signal: release timing, error signatures, affected users.

Service-quality check:
  Question: Are users waiting less, not merely seeing tickets closed?
  Signal: median wait, oldest ticket age, reopens, sampled conversations.

The result is a better decision boundary: add capacity for a short spike, inspect the release, and stop using closure count as the only service claim.

The Repair Loop

Use this loop whenever a model guides a real decision.

1. State the claim. Write the purpose, boundary, variables, assumptions, and prediction before looking at the result.
2. Compare with evidence. Record the observation, its timing, and the uncertainty range. Separate bad data from a bad explanation.
3. Name the mismatch. Ask whether the miss is a boundary problem, stale assumption, missing variable, feedback delay, or proxy failure.
4. Make one useful change. Add, remove, split, or redefine only what can affect the next decision.
5. Write a new prediction. Say what should happen, when it should happen, and what signal would count against the repair.
6. Run a review date. A model without a future check becomes a belief with formatting.
7. Retire when necessary. If the purpose changed, the signals cannot be observed, or another model consistently guides better decisions at lower cost, stop repairing this one.

Repair or Retirement?

Repair when the model still has a useful purpose, its boundary can be stated, and the missing mechanism can be observed well enough to test.

Retire or replace when the decision has changed, the central proxy no longer tracks the outcome, or repeated patches make assumptions impossible to explain. Retirement is a boundary decision about maintenance cost.

Confusion: “A prediction miss proves the model is useless”

Better model:

A miss tells you that at least one claim, boundary, variable, or uncertainty statement needs review. Some parts may still be useful for a narrower question.

Confusion: “More variables always repair a model”

Better model:

Add a variable only when it has an observable signal and changes a decision. Otherwise it increases maintenance cost without improving judgment.

Confusion: “Updating a model means changing the answer to fit the past”

Better model:

The repair must produce a new, risky prediction before the next observation. If nothing could disconfirm it, it is a story, not a testable model.

Confusion: “Keeping two models means refusing to choose”

Better model:

A portfolio assigns each model a question, owner, signal, and decision. Plurality is useful only when it makes action more specific.

Check Your Understanding

Check: A forecast says a queue will shrink, but the queue grows after a product launch. Which first move is strongest?

Think first, then reveal.

Answer: Preserve the original claim and compare its assumptions with the new evidence. Check whether arrival rate changed, whether the boundary excluded the release, and whether the proxy still represents user wait. Do not change capacity numbers before checking those conditions.

Check: A team adds six new variables after every miss. No one can state what each variable predicts. What is the failure?

Think first, then reveal.

Answer: The model is becoming a catalogue rather than a usable representation. Remove variables that have no observable signal or decision role, then write a smaller testable prediction.

Trade-offs and Limits

Repair improves usefulness after conditions change and makes assumptions visible. It costs data collection, monitoring, documentation, and sometimes the discomfort of retiring a familiar explanation.

This helps when:

• the model still supports a real decision;
• the mismatch can be observed and discussed;
• a revised prediction can be tested soon.

It does not protect you from:

• fabricated or incomplete evidence;
• incentives that reward a convenient story;
• changes so rapid that no stable model can be tested;
• decisions that require values or judgment beyond measurement.

You can see the boundary when a model needs a new patch for every observation, when its signals cannot be collected in time, or when two teams use the same term for different outcomes.

The trade-off is explicit:

More detail can improve diagnosis, but it increases maintenance cost.
Faster repair can restore action, but it can hide a deeper boundary problem.
Keeping a model preserves continuity, but retirement may be cheaper and clearer.

Practice: Write a Model Repair Card

Choose a real but bounded situation: a delayed project, a noisy metric, a study habit, a product decision, or a team process. Write one page with:

1. The decision the model should support.
2. The original boundary, variables, proxy, and prediction.
3. One observation that challenged it.
4. The most likely failure category and the evidence for it.
5. One repair or a reason to retire the model.
6. A new prediction, review date, and disconfirming signal.
7. The cost or risk introduced by the change.

Use this rubric:

This card becomes the input for the next lesson's capstone dossier, where you build a model with visible seams for later challenge and repair.

Resources

• [BOOK] The Model Thinker — Focus on model diversity, model limits, and when combining models adds value.
• [BOOK] Thinking in Systems — Focus on feedback, delays, boundaries, and why symptoms can appear far from causes.
• [ARTICLE] Mental Models I Find Repeatedly Useful — Focus on turning a model into a question and a decision rather than a slogan.

Key Takeaways

• A model failure is evidence about a claim, boundary, assumption, variable, proxy, or timing—not an automatic verdict on the whole model.
• Repair means making a specific change, writing a new prediction, and naming what could disconfirm it.
• A small model portfolio can preserve useful views without pretending that every model explains everything.
• Retire a model when its purpose, signals, or maintenance cost no longer justify keeping it.
• A good model leaves visible assumptions and update triggers so another person can challenge it.