Time, Clocks, and Causality

By the end of this lesson, you will be able to...

• explain why a later timestamp does not prove that one event saw another.

• trace causal order through local actions, sent messages, and received messages.

• choose when wall time, monotonic time, or causal metadata is the right evidence.

Idea in one sentence: Distributed systems need more than "what time was it?" because many decisions depend on what information had reached an actor before it acted.

Core Insight

A team uses a private chat room called #launch.

At 10:00, an admin removes Mateo from the room. At 10:01, Mateo's phone sends a message:

"Ship it when the tests finish?"

The server receives the removal first and the message second. The message also has a client timestamp of 09:59 because Mateo's phone clock is slow.

What should the system conclude?

The naive answer is to sort by timestamp. If the message timestamp is before the removal, accept it. If it is after the removal, reject it.

That sounds simple, but it answers the wrong question. The important question is not only "what time did the phone display?" The important question is:

Had Mateo's phone seen the removal before it sent the message?

If the phone was offline, the answer may be no. The message may be concurrent with the removal: not simultaneous, but unordered by evidence.

That does not automatically decide the product policy. A secure chat may reject the message anyway because the server's current membership says Mateo is no longer allowed. But it does change the explanation. The rejection is a permission policy, not proof that Mateo intentionally sent after seeing the removal.

That distinction is the heart of distributed time. A timestamp tells you what a clock said. Causality tells you what information could have influenced a decision.

Plain meaning:

One event is causally after another when information from the first event could have reached the actor that performed the second event.

In this scenario:

Mateo's message is causally after the removal only if his phone received evidence of the removal before sending the message.

Technical name:

The relation is called happens-before. It is the ordering created by local sequence and message flow.

The Naive Idea: Sort Events By Time

Many systems start with a column like this:

event_id | actor        | action          | timestamp
---------+--------------+-----------------+----------
E1       | admin web    | remove Mateo    | 10:00:00
E2       | Mateo phone  | send message    | 09:59:40

Then they write a rule:

earlier timestamp first
later timestamp last

This is tempting because timestamps are familiar. Logs have them. Databases store them. Humans ask questions like "what happened around 10?" A timestamp gives the system a total order.

The problem is that this total order may be fiction.

Wall clocks can drift. They can be corrected forward or backward. A client can send local time, not server time. A server can record when it received an event, not when the user created it.

Even with perfectly synchronized clocks, timestamp order still would not prove influence. If the admin removed Mateo at 10:00 in Madrid and Mateo's phone sent a message at 10:01 while disconnected on a train, the later timestamp does not mean the phone knew about the removal.

So the first repair is not "buy better clocks." Better clocks help with many jobs. They do not answer every ordering question.

Three Kinds Of Time Evidence

Use different clocks for different questions.

A wall clock answers:

What human time did this machine believe it was?

Wall clocks are useful for logs, certificates, token expiration, billing windows, retention policies, and support investigations.

A monotonic clock answers:

How much time elapsed on this one machine?

Monotonic clocks are useful for local durations: timeouts, retry budgets, latency measurements, and "how long have I been waiting?" They should not jump backward when the wall clock is corrected.

Causal evidence answers:

What had this actor already seen before it acted?

It comes from local order and messages. It is the evidence a system needs when replacement, permission, merge, or conflict handling depends on what information crossed a boundary.

For the chat room, the three questions lead to different facts:

wall clock:
  admin event says 10:00
  phone event says 09:59

monotonic clock:
  chat server waited 800 ms for membership state
  phone retry loop waited 5 seconds before reconnecting

causal evidence:
  phone had not received the removal event before sending
  server had received the removal before deciding whether to accept

These facts answer different questions.

Happens-Before

The useful causal rule is small.

An event happens-before another event when one of these is true:

• both events happen in the same process, and the first one occurs earlier in that process;
• the first event sends a message, and the second event receives that message;
• there is a chain of happens-before links connecting them.

If neither event happens-before the other, the events are concurrent.

Concurrent does not mean "same millisecond." It means:

the system has no evidence that either event saw or influenced the other

Look at a causal chain:

admin web:
  A1 click "remove Mateo"
  A2 send removal command -------->

membership service:
                            M1 receive command
                            M2 store member version 18
                            M3 publish removal -------->

chat server:
                                                   C1 receive removal
                                                   C2 reject future Mateo messages

Here, A1 happens-before C2. The order is not proven by timestamps. It is proven by the message path:

A1 -> A2 -> M1 -> M2 -> M3 -> C1 -> C2

Now compare Mateo's offline phone:

Mateo phone:
  P1 compose message
  P2 send message when network returns -------->

membership service:
  M2 store member version 18
  M3 publish removal -------------------------->

chat server:
  C1 receive removal
  C3 receive Mateo message

The server can decide using its policy because it has both pieces of evidence. But the phone's send event is not automatically causally after the removal. There was no path from M2 to P2.

Check: If Mateo's phone received "you were removed from #launch" and then sent the message, would the send still be concurrent with the removal?

Think first, then reveal.

Answer: No. Receiving the removal creates a message path from the removal to the later send. The send is now causally after the removal because the phone had the removal in view before acting.

A Worked Trace

Let's make the evidence visible.

The system has four actors:

admin web
membership service
chat server
Mateo phone

The membership service assigns a version to room membership. The chat server uses that version when deciding whether to accept a message.

Starting state:

room #launch membership version 17:
  Mateo is a member

Mateo phone has seen:
  membership version 17

chat server has seen:
  membership version 17

Now the admin removes Mateo:

membership service:
  input:      remove Mateo from #launch
  transition: version 17 -> version 18
  output:     removal event for version 18

Mateo's phone is offline at that moment. It still has local evidence from version 17:

Mateo phone:
  seen membership version: 17
  action: sends "Ship it when the tests finish?"
  attached evidence: message based on version 17

The chat server receives both events. Here are two possible arrival orders.

Path A: removal arrives first

chat server state:
  before: version 17, Mateo allowed
  receive removal version 18
  after: version 18, Mateo removed
  receive Mateo message based on version 17
  decision: reject or hold for moderation

Path B: message arrives first

chat server state:
  before: version 17, Mateo allowed
  receive Mateo message based on version 17
  decision: accept, or mark as pending until membership catches up
  receive removal version 18
  after: version 18, Mateo removed

The naive timestamp rule tries to decide by comparing 10:00 and 09:59. The causal view asks a richer question:

What membership version had each actor seen?

That gives the system enough information to choose an honest policy.

For a strict private room, the policy might be:

Accept a message only if the chat server has not seen a removal for that user.
If the message is based on an older membership version, reject it or hold it.

For a less strict collaboration tool, the policy might be:

If the sender had not seen the removal yet, keep the message but mark it as sent before membership caught up.

The same causal evidence can support different product rules. Causality does not replace policy. It prevents the policy from pretending a timestamp proved more than it did.

We have moved from:

largest timestamp wins

to:

decide using the evidence each actor had seen

That is the important mental shift.

Logical Clocks And Version Vectors

A logical clock records ordering evidence instead of physical time.

The simplest logical clock is a counter. If all important events pass through one ordered process, each event can get the next number:

17 -> 18 -> 19

That works when one stream owns the order. Distributed systems often have more than one actor, so one counter may hide concurrency. A version vector records what each actor has seen.

In a tiny two-actor example:

membership event:
  {membership: 18, phone: 0}

phone message while offline:
  {membership: 17, phone: 1}

The phone message is behind the membership service on membership, but ahead on phone. Each side has something the other did not include. That is the shape of concurrency.

Now change one thing. The phone receives version 18 before sending:

phone receives removal:
  seen {membership: 18, phone: 0}

phone sends message:
  {membership: 18, phone: 1}

This new message includes the removal history. It is causally after the removal.

The comparison rule is:

• if every counter in A is less than or equal to B, and at least one is smaller, A happened-before B;
• if every counter in B is less than or equal to A, and at least one is smaller, B happened-before A;
• if each version is ahead in at least one place, they are concurrent.

Version vectors cost storage, complicate APIs, and can grow when there are many actors. But they make an invisible question inspectable:

does this event include the history of that event?

Check: Compare {membership: 18, phone: 0} with {membership: 17, phone: 1}. Is either one causally after the other?

Think first, then reveal.

Answer: No. The first is ahead on membership history. The second is ahead on phone history. Each has evidence the other lacks, so they are concurrent.

Trade-offs And Limits

The trade-off is that causal evidence gives better decisions, but it makes the system heavier.

It improves:

• conflict detection;
• audit explanations;
• merge decisions;
• permission and replacement rules that depend on what an actor had seen.

It costs:

• metadata on messages or records;
• comparison logic;
• storage for unresolved conflicts;
• product decisions for cases where events are concurrent.

It can still fail when the policy is unclear. A system can correctly detect concurrency and still make a bad product choice. The causal metadata tells the truth about order. It does not decide the business rule.

It also does not remove the need for physical time. Wall clocks still matter for expiration, leases, certificates, billing, retention, and incident timelines. Monotonic clocks still matter for timeouts and latency measurement.

You can see the boundary when teams argue about words like "latest," "stale," or "after." If the real question is "what had this actor seen?", a timestamp-only design is probably too weak.

A useful design test is to ask what harm comes from being wrong. If the harm is a confusing log line, wall time may be enough. If the harm is deleting work, accepting an old permission, or explaining a false audit trail, the system needs stronger ordering evidence.

Common Confusions

Confusion: Concurrent means simultaneous

Why it is tempting:

The word sounds like two things happened at exactly the same time.

Better model:

Concurrent means unordered by causal evidence. Two events can be minutes apart and still be concurrent if no information path connected them before they occurred.

Confusion: Better clock synchronization solves causality

Why it is tempting:

If all clocks are close, timestamp order feels trustworthy.

Better model:

Synchronized clocks help logs and deadlines. They do not prove what a participant had seen. A disconnected phone can act after a removal in physical time without knowing about the removal.

Practice

Choose one workflow:

profile photo update
shopping cart item addition
feature flag activation
password reset
room membership change

Fill in this review:

actors:
state being changed:
event A:
event B:
what each actor had seen before acting:
causal evidence needed:
safe policy if the events are concurrent:
safe policy if one event happens-before the other:
where wall time is still useful:

Model answer for password reset:

actors:
  user browser, auth service, email service

state being changed:
  active password reset token

event A:
  user requests a new reset token

event B:
  user clicks an older reset link

what each actor had seen before acting:
  the browser may only have the old email;
  the auth service may have issued a newer token

wall-clock fact that may be misleading:
  the old email can be opened after the new token was issued

causal evidence needed:
  which token generation the click refers to

safe policy:
  accept only the currently active token;
  invalidate older generations after a newer token is issued

where wall time is still useful:
  token expiration and support investigation

Resources

• [PAPER] Time, Clocks, and the Ordering of Events in a Distributed System - Focus: The original happens-before relation and logical clocks.
• [ARTICLE] Why Logical Clocks are Easy - Focus: Practical intuition for logical clocks and causal order.
• [BOOK] Designing Data-Intensive Applications - Focus: Replication, conflict detection, and the limits of timestamp ordering.

Key Takeaways

• A wall-clock timestamp says what time a machine believed it was; it does not prove what the actor had seen.
• Happens-before order comes from local sequence, sent messages, received messages, and chains of those links.
• Concurrent events are unordered by evidence, not necessarily simultaneous.
• Logical clocks and version vectors make causal evidence visible enough to compare histories.
• Causal metadata helps policies tell the truth, but the product still chooses how to handle rejection, merge, conflict, or coordination.