Probability Begins by Naming What Could Happen

By the end of this lesson, you will be able to...

• Define an experiment, outcome, sample space, event, and probability model in plain language.

• Build a small sample space whose outcomes do not overlap and cover every case inside a chosen boundary.

• Explain why a correct calculation can still answer the wrong question when the model leaves out an important outcome.

Idea in one sentence: Before probability can assign numbers, we must decide what can happen and what question our model is meant to answer.

Core Insight

A notification service sends a message to a user.

The client waits two seconds for an acknowledgement. If no acknowledgement arrives, the client retries once. The team wants to answer a simple question:

What is the probability that the client eventually receives an acknowledgement?

It is tempting to start with arithmetic. Perhaps the dashboard says that 96% of requests receive an acknowledgement. We could write \(P(\text{ack}) = 0.96\) and move on.

But move on to what?

The number does not yet tell us what one run of the process looks like. It does not say whether an acknowledgement arrived on the first attempt or after a retry. It does not say whether two timeouts count as one failure or two failures. It does not even say whether "acknowledged" means "the message reached the user" or only "the client received a response."

Probability begins before the number.

It begins when we describe the possible results of one uncertain process. That description is a model choice. If the possible results are vague, overlapping, or incomplete, later calculations can be perfectly correct and still be useless.

The Question Chooses the Boundary

Our first question is about what the client observes during one request. We will therefore model the client-side acknowledgement pattern.

We will not yet model:

• Whether the user reads the message.
• Whether the server processes a request after the client times out.
• The exact response time in milliseconds.
• Which machine handles each attempt.

Those details may matter for other questions. They do not all need to appear in this first model.

This is not carelessness. It is a boundary.

A model must leave details out, or it becomes as difficult to inspect as the real system. The trade-off is that a smaller model answers fewer questions. We should choose the smallest boundary that still supports the decision we need to make.

For this lesson, one run starts when the client sends the first attempt and ends when either:

1. An acknowledgement arrives, or
2. The second attempt reaches its timeout.

Now the uncertain process has a clear start, a clear stopping rule, and a clear observation boundary.

From a Process to Possible Outcomes

Let:

• A mean "an acknowledgement arrives before the current deadline."
• T mean "the current attempt reaches its timeout without an acknowledgement."

The client stops after an A. It retries only after the first T. That gives us three possible complete histories:

Notice what is not in the table: AA.

The client stops after the first acknowledgement, so a second attempt is never sent in that branch. AA is not a possible complete history under this procedure.

Each row is an outcome.

Plain meaning:

An outcome is one complete result of the uncertain process at the level of detail we chose.

In this scenario:

TA is an outcome because it describes one complete client-visible history from start to stopping point.

Technical name:

The set of all possible outcomes is the sample space. It is often written with the Greek letter omega:

The sample space is not "everything reality could possibly do." It is every result represented by this model under this boundary.

Two Tests for a Useful Sample Space

A finite sample space should pass two basic tests.

Test 1: The outcomes do not overlap

One run must land in exactly one row.

If the first attempt is acknowledged, the result is A, not also TA. If the first attempt times out and the retry is acknowledged, the result is TA, not also TT.

This property is called mutual exclusivity.

Test 2: The outcomes cover every represented case

Under our stopping rule, every run ends as A, TA, or TT.

There is no fourth client-visible acknowledgement pattern. The three outcomes are therefore exhaustive inside the model.

These tests protect later arithmetic. If two outcomes overlap, we may count one run twice. If an outcome is missing, the probabilities may describe only part of the process.

Events Collect Outcomes That Answer a Question

An outcome is one complete result. An event is a set of outcomes that share a property we care about.

Consider the event:

Which outcomes make \(E\) happen?

Now consider:

The retry is used after a first timeout:

The two events overlap at TA. A run with outcome TA both receives an eventual acknowledgement and uses the retry.

This distinction matters:

• TA is an outcome: one complete result.
• \(\{A, TA\}\) is an event: several outcomes grouped to answer a question.
• \(\{TA, TT\}\) is another event: a different grouping of the same sample space.

We do not need a new sample space for every question. We can ask different questions by defining different events over the same outcomes.

A Tiny Probability Model

A sample space lists what the model allows. A probability model also assigns a probability to each outcome.

Suppose a historical sample of comparable requests suggests this simplified model:

The numbers obey three basic rules:

1. No outcome has a negative probability.
2. The probabilities of all outcomes add to 1.
3. For outcomes that do not overlap, the probability of their union is the sum of their probabilities.

Check the total:

Now calculate the probability of eventual acknowledgement:

The probability that a retry is used is:

The worked path is:

1. Start with the operational question.
2. Choose one request as the experiment.
3. Define the stopping rule.
4. List the complete outcomes: A, TA, and TT.
5. Group outcomes into the event \(E = \{A, TA\}\).
6. Assign probabilities to outcomes.
7. Add the probabilities inside \(E\).
8. Conclude that \(P(E) = 0.96\).

The naive path starts with "96% succeed" and hides the model. The stronger path shows what 96% means, which outcomes it combines, and which observation boundary supports it.

So far, we have not derived the three outcome probabilities. We accepted them as a toy model based on comparable historical runs. Later lessons will ask where such numbers come from, how samples vary, and when dependence makes a simple estimate unsafe.

What the Model Can Answer

Our model can answer client-visible questions such as:

• What is the probability of receiving an acknowledgement?
• What is the probability of using the retry?
• What is the probability that both attempts time out?

It cannot answer:

• What is the probability that the user actually receives exactly one message?
• How long does a successful request take?
• What is the probability of duplicate server processing?
• Does a timeout mean the server failed?

Those questions need different outcomes or extra variables.

For example, TT means only that the client observed two timeouts. The server might have processed the first request and sent a response that arrived too late. A timeout is an observation at the client boundary, not proof of server failure.

This is the first important limit of probability modeling:

A probability answers a question inside a model. It does not repair a model that represents the wrong thing.

Granularity Is a Trade-off

We could use a coarser sample space:

This model is easy to communicate. It answers the eventual-acknowledgement question directly.

But it cannot tell us how often the retry runs. It hides the difference between A and TA.

We could also use a much finer sample space. We might record:

• Which attempt the server processes.
• Whether the message is delivered once or twice.
• Whether the acknowledgement is late or lost.
• The response-time bucket.
• The server region.

That richer model can answer more questions. It also costs more to define, measure, estimate, and explain. Many cells may have too little data for stable probability estimates.

The trade-off is not "simple is good" versus "detail is good." The useful question is:

What is the least detail that keeps the decision-relevant outcomes distinct?

Use a coarse model when its grouped outcomes behave the same for the current decision. Split an outcome when the hidden differences would change the decision.

The signal that the boundary is too coarse is repeated surprise: the model predicts the event rate correctly, but the team still cannot explain retry load, duplicate work, tail latency, or user impact.

Common Confusions

Confusion: Every outcome must be equally likely

Why it is tempting:

Coins and dice are common first examples, and their outcomes are often modeled with equal probabilities.

Better model:

A sample space lists possible outcomes. It does not say they are equally likely. Our outcomes have probabilities 0.80, 0.16, and 0.04.

Confusion: An event is one outcome

Why it is tempting:

Everyday language uses "event" for one occurrence.

Better model:

In probability, an event is any set of outcomes, including a set with one outcome. "Eventually acknowledged" contains both A and TA.

Confusion: The sample space is discovered, not designed

Why it is tempting:

The real process exists before we model it.

Better model:

Reality constrains the model, but the question chooses the boundary and granularity. A client-observation model and a message-delivery model can describe the same request differently.

Confusion: A missing case can be fixed by changing the probabilities

Why it is tempting:

Probabilities feel like the adjustable part of a model.

Better model:

If a meaningful outcome is absent, no reassignment of probability among the existing outcomes can represent it. First repair the sample space. Then estimate probabilities.

Check Your Understanding

Check: Is \(\{TA, TT\}\) an outcome or an event?

Think first, then reveal.

Answer: It is an event. It groups two complete outcomes under the property "the retry is used."

Check: The server may deliver the message after the client records TT. Does that make TT impossible?

Think first, then reveal.

Answer: No. TT is a client-visible outcome: both attempts reached their timeout. It does not claim that the server performed no work or that delivery was impossible. Answering the delivery question requires a richer model.

Practice: Model a Batch Job

A batch job has one deadline. Each run ends in one of three observed states:

• F: it finishes before the deadline.
• L: it finishes after the deadline.
• X: it fails without finishing.

In 200 comparable historical runs, 150 were F, 30 were L, and 20 were X.

Do the following:

1. Write the sample space.
2. Define the event \(S\): "the job meets its deadline."
3. Define the event \(C\): "the job eventually completes."
4. Assign a simple empirical probability to each outcome.
5. Calculate \(P(S)\) and \(P(C)\).
6. Name one question this model cannot answer.

Model answer

The empirical probabilities are:

Therefore:

The model cannot answer how late an L run is, why an X run fails, or whether tomorrow's workload is comparable to the historical runs. Any one of those is a valid boundary statement.

Connections

The next lesson, Random Variables Turn Outcomes into Distributions, will add a numeric quantity to each outcome. Instead of asking only whether an acknowledgement arrives, we will be able to model values such as attempt count, latency, cost, or number of failures.

Later lessons will challenge the probability assignments themselves. They will ask how samples vary, how dependence changes evidence, and how model assumptions fail under drift, bursts, or hidden state.

Resources

• [COURSE] MIT OpenCourseWare: Sample Space — Focus: See how an uncertain experiment becomes a precise set of possible outcomes.
• [COURSE] MIT OpenCourseWare: Sample Space Examples — Focus: Compare several choices of outcome boundary and granularity.
• [REFERENCE] MIT 18.05: Probability Terminology and Examples — Focus: Review experiment, sample space, event, probability function, and small finite examples.

Key Takeaways

• Probability starts by defining one uncertain experiment and the complete outcomes represented by the model.
• A useful sample space has outcomes that do not overlap and that cover every case inside the chosen boundary.
• An event is a set of outcomes selected to answer a question.
• Probability assignments must be non-negative and sum to 1 across a finite sample space.
• Correct arithmetic cannot rescue a sample space that omits a decision-relevant outcome.