Hidden State Gives the Past a Memory

By the end of this lesson, you will be able to...

• Separate an unobserved system state from the signals it emits.

• Use a small transition model to explain why recent history changes a forecast.

• Check whether a proposed state representation is useful, observable, and honest about uncertainty.

Idea in one sentence: A process can remember its past through a hidden state, even when the current measurement looks like an ordinary random draw.

Core Insight

Consider an API whose dashboard shows one request at a time. Most requests finish in 100 ms, but during some incidents latency rises to 800 ms and errors cluster for several minutes. An engineer asks, “Given that the last request succeeded, will the next one succeed?”

The last observation is useful, but it is not the whole system. The service may be in a healthy mode or a degraded mode caused by a saturated database connection pool. A healthy request can occur during a degraded episode, and a slow request can occur while the system is recovering. The cause is not directly visible in the request result.

Use two variables:

• \(S_t\): the hidden operating state at time \(t\), such as healthy or degraded.
• \(Y_t\): the observed signal at time \(t\), such as success, timeout, or latency bucket.

The state evolves over time and produces observations:

The observations are noisy evidence about the state. Once the state is included, the apparent memory in the request sequence has a mechanism: degraded systems tend to remain degraded for a while.

The Pattern We Want to Name

Lesson 012 showed that a time series can have dependence. A hidden-state model explains one important source of that dependence:

1. A state changes according to transition probabilities.
2. The current state generates an observation according to an observation rule.
3. The state is not fully visible, so the observations leave us uncertain about it.

The tempting but weak model is:

It treats every request as an independent snapshot. A stronger model says that observations become more predictable once we condition on the state:

and the state itself has a compact memory rule:

This is the Markov property for the chosen state representation. It does not claim that real systems have no history. It claims that the state contains enough of the relevant history for the next transition.

A Tiny Example

Let \(S_t\) be either \(H\) (healthy) or \(D\) (degraded). Suppose the transition table is:

The degraded state is persistent: once the system is degraded, it remains degraded with probability 0.60 in the next interval.

Let the observation \(Y_t\) be an error indicator. Assume:

An error is much more likely in state \(D\), but it is not impossible in \(H\). That is why one observation cannot identify the state with certainty.

Suppose before seeing a request we estimate \(P(D)=0.20\). After observing an error:

The error raises our belief that the service is degraded from 0.20 to about 0.76. This is conditional-probability bookkeeping, not a claim that the state is known.

To forecast the next state, propagate that uncertainty through the transition table:

After one error, the next interval has roughly a 48% chance of being degraded under this model. A model that ignored the state would keep using the unconditional rate and miss the persistence.

The Formal Definition

A simple hidden-state process has four pieces:

1. A set of states \(\mathcal S\), such as \(\{H,D\}\).
2. An initial distribution \(\pi(s)=P(S_0=s)\).
3. A transition matrix \(T\), where

1. An observation or emission rule \(E\), where

Every row of \(T\) sums to one. The observation probabilities for a fixed state also sum to one across all possible signals.

The Markov assumption is useful because it turns an indefinitely long history into a current state distribution. If our current belief is a vector \(b_t\), then a prediction step is:

An observation then reweights the states according to their emission probabilities. In practice, the exact calculation may be implemented by a monitoring system, but the conceptual loop is simple:

The state does not need to be physically hidden forever. It may be a variable that is expensive, delayed, or impossible to measure directly, such as “dependency is close to saturation” or “cache contents are mostly cold.”

Working Through a Trace

Suppose an API emits this sequence over five intervals:

The two timeouts should not be treated as two unrelated bad draws. They jointly support a persistent-state explanation. A single success at time 5 should lower, but not erase, the probability of degradation because recovery may be partial and the emission rule is noisy.

This also explains why a useful alert often includes a short history instead of only the latest point. A state estimate can combine recent errors, queue depth, database saturation, and deployment phase. Those signals may be conditionally independent given the state only as an approximation; the model should say what it assumes.

The representation matters. If “degraded” combines database saturation, network loss, and a bad deployment, its transition probabilities may be unstable and its observations inconsistent. Splitting it into too many states creates a model that is hard to estimate and hard to operate. A small state model is valuable when it compresses relevant history without pretending that unrelated causes are identical.

What This Lets Us Predict

Hidden-state reasoning supports several decisions:

• Early warning: several weak signals can raise the probability of a degraded state before a hard failure.
• Recovery: a success after an incident may indicate one healthy observation, not a fully recovered system.
• Sampling: collect state-relevant signals at a resolution that can see transitions; daily averages may erase a ten-minute degraded episode.
• Intervention: choose a mitigation that changes the transition probabilities, such as draining a bad host or reducing load.
• Evaluation: test whether the state model predicts held-out observations better than an independent-snapshot baseline.

The model also makes a trade-off explicit: adding states can improve explanatory precision, but it increases estimation cost, monitoring complexity, and the chance of confusing noise with a new regime. Start with the smallest state distinction that changes the operational decision.

Edge Cases and Counterexamples

The wrong state can hide memory

Suppose the only state variable is “CPU percentage.” If latency also depends on cache warmth, database locks, and queue age, the proposed state is incomplete. The next observation may still depend on older history, so the Markov property fails for that representation.

A state can be observable but delayed

Database saturation may be visible only after a metric arrives 30 seconds late. Calling it “hidden” describes the decision timing, not the physics. A delayed signal can still be useful if the model records its lag.

Transition probabilities can change

A deployment, traffic mix, or capacity change may alter \(T\). A model fitted before the change can become stale while still producing confident forecasts. Recheck transitions after interventions.

Similar observations can come from different states

A timeout could mean a slow database, a network partition, or a client cancellation. If those causes require different actions, one observation category is too coarse. Add a signal or split the states only when the distinction is actionable.

Hidden state is not automatically a cause

The state is a useful summary of conditions that predict observations. Naming a state “degraded” does not prove why it became degraded. Causal investigation still needs interventions or independent evidence.

Common Confusions

Confusion: Markov means the system has no history

Better model: the chosen state is a compressed history. The future can depend on the past through \(S_t\), even when it does not need every older observation separately.

Confusion: One observation identifies the state

Better model: observations have emission probabilities. A signal changes state probabilities; it rarely makes them exactly zero or one.

Confusion: A hidden state is the same as a latent variable model

Better model: this lesson uses a small transition-and-observation model for operational intuition. Full latent-variable inference and Bayesian model comparison belong in a deeper track.

Confusion: More states always improve the model

Better model: extra states can overfit sparse evidence and make transitions impossible to estimate. Add a state only when it changes prediction or action.

Practice: Infer a Degraded Episode

A worker service has two hidden states: healthy \(H\) and degraded \(D\). Its transition probabilities are:

Timeout probabilities are:

Before an observation, \(P(D)=0.25\). A timeout occurs.

1. Is the timeout more consistent with \(H\) or \(D\), and why?
2. Why should the next interval still be treated as uncertain?
3. Name one additional signal that could distinguish database saturation from network loss.
4. What would be a bad response to one successful request immediately after the timeout?

Model answer

The timeout is more consistent with \(D\), because its likelihood is 0.30 in \(D\) and only 0.02 in \(H\). It is not proof: healthy systems can also time out, and the transition model leaves uncertainty about recovery. A useful next signal could be database pool wait time, network retransmits, or per-dependency latency, chosen according to the competing explanations. A bad response would be to declare the incident over after one success; the state may remain degraded with probability 0.70, and a single observation is noisy evidence.

Connections

Lesson 012 introduced time dependence, drift, and autocorrelation. This lesson explains one mechanism that creates persistence: an unobserved state that transitions slowly and emits noisy signals. Lesson 014 will connect that stateful dependence to queues, bursts, locality, and finite capacity. The track stops at small, inspectable state models rather than moving into full hidden Markov-model inference.

Resources

• [COURSE] MIT OpenCourseWare: Introduction to Probability - Focus: Practice conditional probability and state-based reasoning with small examples.
• [REFERENCE] Stanford Encyclopedia of Philosophy: Markov Process - Focus: Use the Markov property as a conceptual guide to memory and state representation.

Key Takeaways

• A hidden state can carry relevant history even when each observation is noisy.
• Transition probabilities describe how operating conditions persist or recover; emission probabilities describe what the instrument can reveal.
• A Markov model is only as good as the state representation: it must retain the history that changes the next prediction.
• State probabilities are beliefs, not direct observations or causal explanations.
• Prefer the smallest state model that improves a forecast or changes an operational action.