Cache Invalidation and Freshness Control

Day 068: Cache Invalidation and Freshness Control

Caching gets hard the moment the source changes, because now the real question is not storage but who is still allowed to believe the old answer, for how long, and at what cost.

Today's "Aha!" Moment

It is easy to think of invalidation as a technical cleanup step: delete the key when the value changes. That description is too narrow. What you are really controlling is freshness. When the source of truth changes, you must decide how quickly readers should stop seeing the old answer, and how much complexity you are willing to pay to achieve that.

Use one concrete example: the learning platform changes a course price from 49 to 59. If the public catalog page shows 49 for another minute, that may be tolerable. If checkout still believes 49 while payment logic believes 59, that may be unacceptable. Both are "stale cache" situations, but the business meaning is different. That is why invalidation cannot be designed with one blanket rule.

That is the aha. Cache invalidation is not mainly about deletion. It is about controlling the spread of newer truth through old copies. TTL, explicit deletion, versioned keys, validation, and stale-while-revalidate are all different answers to the same question: how do readers move from the old answer to the new one?

Once you see it this way, the hard parts line up cleanly. First decide how stale is acceptable for this path. Then decide how caches learn about change. Then decide what happens when many requests miss or refresh at once. Those are the three real design axes.

Why This Matters

The problem: A fast cache that serves the wrong answer at the wrong moment can be worse than a slower system that stays semantically clear and predictable.

Before:

After:

Real-world impact: Fewer confusing stale reads, safer product behavior around important updates, and much lower risk of cache stampedes turning one hot key into an outage.

Learning Objectives

By the end of this session, you will be able to:

  1. Explain invalidation as freshness control - Connect stale reads to source changes, multiple copies, and business meaning.
  2. Compare how caches learn about change - Distinguish TTLs, explicit invalidation, and version-based strategies conceptually.
  3. Reason about misses under load - Understand why refill behavior and stampede prevention are part of the same design problem.

Core Concepts Explained

Concept 1: Freshness Budgets Come Before Invalidation Mechanisms

The first design question is not "Which invalidation pattern should we use?" It is "How wrong is too wrong, for how long, on this path?" That is a freshness budget.

For example:

This matters because the same invalidation mechanism can be perfectly fine for one class of data and clearly inadequate for another. A two-minute TTL is not good or bad in the abstract. It is only good or bad relative to what readers are allowed to believe during those two minutes.

source changes
   -> how long may readers still see old data?
      -> that answer drives the strategy

This is the biggest teaching trap in caching. Teams jump straight to technology and skip the semantic question. But without a freshness budget, invalidation is just a random operational habit.

The trade-off is simplicity versus precision. Broad TTLs are easy, but they only work where bounded staleness is genuinely acceptable.

Concept 2: TTL, Explicit Invalidation, and Versioning Are Different Ways for Caches to Learn About Change

Once the freshness budget is clear, you can choose how the cache learns that truth has changed.

TTL is the simplest. The cache keeps the value until the timer runs out:

cache entry created
   -> time passes
      -> entry expires
         -> next read fetches fresh data

That works well when the stale window is acceptable and simplicity matters. But it does not react to the change itself. It only reacts to elapsed time.

Explicit invalidation reacts to the update. When the source changes, the system deletes or refreshes affected keys:

def update_course_price(db, cache, course_id, new_price):
    db.update_course_price(course_id, new_price)
    cache.delete(f"course:{course_id}:price")

That is stronger on freshness, but it also means you must know which keys and derived views depend on the change. That can be straightforward for one object key and much harder for list pages, aggregates, or precomputed responses.

Versioning takes another approach: instead of overwriting the old answer, it publishes a new edition and lets readers move to the new key or version. This is often useful when immutable assets or generated content should coexist briefly with older references still in flight.

The trade-off across all three is the same one in different forms:

No one of them is "the correct invalidation strategy" universally. They are just different change-propagation policies.

Concept 3: Refill and Stampede Control Are Part of Freshness Design Too

Many teams think invalidation ends once the old key is gone. In practice, the next problem starts exactly there: what happens when readers come back and the cache is empty?

Take a hot course page under traffic. The key expires or is invalidated, and thousands of requests arrive within seconds. If every one of them independently recomputes the same answer, the source of truth gets hit by a burst much larger than normal load. The cache that was protecting the system suddenly becomes the reason the system spikes.

That is why refill behavior belongs inside the invalidation conversation:

value becomes stale or disappears
   -> many readers miss
      -> who is allowed to refill?
      -> who waits?
      -> who gets stale data briefly?

Common strategies include:

This is where freshness and traffic shaping meet. The system is not only deciding how current data should be. It is also deciding how much synchronized load the source can survive when that data stops being current.

The trade-off is added complexity versus origin protection. For hot keys and bursty traffic, refill design is often what turns a workable cache into a production-safe cache.

Troubleshooting

Issue: Choosing TTLs by habit instead of by the meaning of stale data.

Why it happens / is confusing: It is easy to pick round numbers like 60 seconds or 5 minutes without asking what stale data means for the actual workflow.

Clarification / Fix: Define freshness budgets per read path first. Then choose TTLs or invalidation rules that match those budgets instead of cargo-culting one number.

Issue: Invalidating the obvious key but forgetting derived or aggregated views.

Why it happens / is confusing: Teams often remember the primary object key but forget list views, summary pages, or precomputed response caches built from that object.

Clarification / Fix: Map the full read surface affected by a change. The stale answer may be in the object cache, the list response, the summary page, and the CDN all at once.

Advanced Connections

Connection 1: Invalidation ↔ Consistency Design

The parallel: Invalidation is a consistency question in practical form because it determines how quickly different readers stop seeing old state.

Real-world case: Product descriptions, prices, inventory, and entitlements all impose different costs for stale reads, which is exactly why they need different freshness budgets.

Connection 2: Invalidation ↔ Traffic Shaping

The parallel: The way entries expire or refresh changes not only correctness but also how load returns to the source of truth.

Real-world case: Jittered TTLs, request coalescing, and stale-while-revalidate are often more about protecting the origin under bursty traffic than about raw cache hit rate.

Resources

Optional Deepening Resources

Key Insights

  1. Invalidation starts with freshness budgets - The hard part is deciding how stale is acceptable on each read path.
  2. Strategies differ in how they learn about change - TTL, explicit invalidation, and versioning are different propagation policies, not just different implementation details.
  3. A missing key is still part of cache design - Refill behavior under load can be as important as stale-read behavior.

Knowledge Check (Test Questions)

  1. What is the best first question in cache invalidation design?

    • A) How stale may this read path safely be before the behavior becomes wrong or confusing?
    • B) Which cache product should we standardize on?
    • C) How short can we make the TTL everywhere?

  2. What does explicit or event-based invalidation improve over TTL alone?

    • A) It reacts closer to the actual source change when waiting for time-based expiry would leave data stale too long.
    • B) It guarantees no cache miss will ever happen again.
    • C) It removes the need to think about all derived caches.

  3. Why is refill strategy part of invalidation design?

    • A) Because when a popular key expires or is deleted, many readers may return at once and overload the source unless refresh is controlled.
    • B) Because once a key expires, the cache no longer affects system behavior.
    • C) Because refill strategy only matters for write-heavy systems, not read-heavy ones.

Answers

1. A: Invalidation strategy should begin with the business meaning of stale data, because that determines how strong or simple the freshness mechanism can be.

2. A: Explicit invalidation helps when stale windows need to track real updates more closely than a timer would allow.

3. A: Expiry and deletion are only half the story. What happens immediately after the miss can determine whether the source stays healthy under load.

← Back to Learning