CAP, PACELC, and Partition-Time Behavior

By the end of this lesson, you will be able to...

• explain the CAP choice for one operation during a network partition.

• compare partition-time behavior with normal-day PACELC latency costs.

• design response wording that matches the evidence behind a distributed decision.

Idea in one sentence: CAP asks what promise survives when replicas cannot talk, and PACELC asks what latency you pay for stronger evidence even when they can.

Core Insight

A social app lets people claim a public username.

The promise is simple:

one username -> one official account

If Maya owns @luna, no one else should receive a valid confirmation for @luna.

The app runs in two regions, Madrid and Dublin. Both regions keep a copy of the username registry so signups can feel fast. At 10:00, @luna is available in both copies. Then the network link between the regions breaks.

At 10:01, Maya signs up through Madrid and asks for @luna. At 10:01, Noah signs up through Dublin and asks for @luna.

Both regions are alive. Both have disks. Both can answer local users. But each region has lost the evidence it needs to know what the other region is doing.

The naive design says:

If the local copy says the username is available, confirm it now.
Replicate the update later.

That design keeps the signup form moving. It also creates a product lie. After the link heals, the system may discover two official confirmations for one username. A support team can apologize, rename one account, or compensate someone. It cannot make both original confirmations true.

Plain meaning:

During a partition, a distributed system may have to choose between local success and one shared answer.

In this scenario:

Madrid and Dublin cannot both confirm @luna independently while also preserving the promise that one username has one official owner.

Technical name:

This is the useful pressure behind CAP. PACELC adds the normal-day question: even when the link works, how much latency do you spend to get stronger confirmation?

Start With The Promise

CAP is easiest to misuse when it starts as an acronym. Start with the operation.

operation:    claim username @luna
state:        username registry
actors:       Madrid signup, Dublin signup, registry replicas
promise:      a confirmed username belongs to one account
failure:      Madrid and Dublin cannot exchange messages
success:      no two users receive valid confirmation for @luna

The state is not "the whole database." It is this piece of state and this promise. A profile bio might use a different rule. Search results might use a different rule. The handle claim is strict because two accepted outcomes cannot both be honored.

The question is:

What evidence lets this region safely say "the username is yours"?

If the answer requires fresh information from the other region, a partition removes that evidence. The region can still reply. It just cannot honestly make the same claim.

The Naive Design

The tempting design gives each region local authority:

Madrid registry copy:
  @luna = available

Dublin registry copy:
  @luna = available

rule:
  if local copy says available, confirm immediately
  send replication event later

This design has a real benefit. Most signups return quickly. A nearby region can answer without waiting for a distant round trip. The code path is easy to explain.

It breaks when communication is missing at the exact moment two isolated regions change the same fact.

Madrid:
  sees @luna available
  confirms @luna for Maya

Dublin:
  sees @luna available
  confirms @luna for Noah

after the link heals:
  @luna has two confirmed owners

The problem is not a bad timestamp. It is not that either region crashed. The problem is conflicting authority. Each side acted as if its local evidence was enough to make a global promise.

Check: Could the system repair this by choosing the earlier timestamp after the link heals?

Think first, then reveal.

Answer: It can choose a winner, but it cannot make both confirmations true. Timestamp repair may produce a final database value, but it does not preserve the promise made to the user who loses the name.

A Worked Trace

Let's compare two partition-time policies.

Starting state:

global registry version 52:
  @luna = available

Madrid has seen:
  version 52

Dublin has seen:
  version 52

Then the network partitions:

Madrid  ---- X ----  Dublin
  Maya                  Noah
  claim @luna           claim @luna

Path A: Protect One Official Answer

In this design, a region may not confirm a username unless it reaches the registry owner or a quorum that can make one official decision.

Madrid input:
  Maya wants @luna

Madrid transition:
  local copy says available
  required evidence: owner or quorum confirmation
  partition prevents confirmation

Madrid output:
  "We cannot confirm this username right now."

Dublin follows the same rule. Both regions may be healthy, but neither can safely issue the final confirmation while isolated.

This path protects the promise. It gives up CAP availability for this operation because a non-failing node does not return a successful claim. A product may still show a useful page, save an unconfirmed draft, or ask the user to retry. But the protected operation, "claim @luna now," is not available in the CAP sense.

Path B: Keep Local Claims Moving

In this design, each region may confirm from its local copy and reconcile later.

Madrid:
  @luna available at version 52
  confirm @luna for Maya at local version 53M

Dublin:
  @luna available at version 52
  confirm @luna for Noah at local version 53D

after the link heals:
  53M and 53D conflict

This path is available to local requests. It is also divergent. The later repair can pick a winner, but the system has already created two official-looking answers.

The intermediate state is the lesson. During the partition, each region has partial knowledge. A local copy is not enough evidence for a global uniqueness promise.

What CAP Actually Names

CAP is not "choose any two letters." In a real distributed deployment, partitions are not optional. The practical question is what a specific operation does while communication is broken.

Consistency, in CAP, means the operation behaves as if there is one current, coherent answer. For the username claim, @luna cannot be confirmed for both Maya and Noah.

Availability means a request to a non-failing node receives a successful response. A polite "try later" response may be good product behavior, but it is not a successful username claim.

Partition tolerance means the system has defined behavior when messages between groups are lost, delayed too long, or delivered only one way.

For this operation, the decision rule is:

If both isolated sides accept the operation,
can the two accepted outcomes both remain true?

yes -> local progress and later merge may be acceptable
no  -> require an owner, a quorum, or a refusal path

A shopping cart addition often has a real merge rule: keep both added items. A notification count can often add numbers later. A public username, a password reset token, a payment ledger entry, or a production feature flag may need one official authority because two accepted answers would break the promise.

PACELC: The Normal-Day Price

CAP focuses on the partition scene. PACELC adds the ordinary scene.

The letters are a compact reminder:

P: if there is a partition,
   choose between availability and consistency

E: else, when the network is healthy,
   choose between latency and consistency

For the username service, compare two healthy-link paths.

stronger confirmation:
  user -> Madrid -> registry owner/quorum -> Madrid confirms
       waits for remote evidence

lower-latency response:
  user -> Madrid reserves locally -> Madrid replies quickly
       remote registry catches up later

The stronger path costs latency. It waits for a remote owner, quorum, or coordination path before saying "confirmed." The lower-latency path feels better, but its response must be narrower. It might safely say:

"Request received. Username is pending confirmation."

It should not say:

"@luna is yours."

unless the system has the evidence to defend that claim.

The trade-off is not only technical. It changes product language. Words like "saved," "reserved," "confirmed," "visible," and "published" should match the evidence behind the write.

Check: If Madrid writes a local pending reservation and returns "confirmed globally," what is wrong?

Think first, then reveal.

Answer: The response overstates the evidence. Local durability may be true, but global confirmation requires stronger evidence, such as the owner or quorum accepting the claim.

Response Words Are Part Of The Design

A distributed write often has several honest response levels.

received:
  the request reached this region

saved locally:
  this region durably stored the request

pending:
  the system still needs remote evidence before making the final promise

confirmed:
  the authority for this operation accepted the change

visible everywhere:
  the change has reached every place the product promise names

These words are small, but they carry the user contract. If the signup API returns 201 Created with username_confirmed: true, it is making a stronger claim than an API that returns 202 Accepted with status: pending_confirmation.

This is where CAP and PACELC become design tools instead of slide labels. The team is not only choosing a database mode. It is choosing what each response means, what evidence is required for that response, and what the user should see when the evidence is unavailable.

Trade-offs And Limits

The trade-off is that stronger confirmation protects the promise, but spends latency and may reject work during uncertainty.

This helps when accepted outcomes cannot both be true. It costs coordination, extra waiting, more failure handling, and a degraded path for partitions.

The local-progress design helps when users value immediate response and the domain has a real merge or compensation rule. It costs conflict handling and may create user-visible repair.

CAP and PACELC do not choose the answer for the whole product. They force the team to make the promise explicit. One application can use strong confirmation for usernames, local merge for cart additions, asynchronous indexing for search, and stale reads for public profile views.

A good design writes these choices down per operation. Otherwise, the default storage behavior silently becomes the product promise, especially during regional failure.

Useful signals make the boundary visible:

• partition duration between regions;
• number of username claims waiting for confirmation;
• latency of owner or quorum confirmation;
• conflicts found after replication catches up;
• number of user-visible compensations or forced renames; and
• responses whose wording does not match their evidence.

If conflicts are rising, the system is accepting work it cannot safely merge. If pending claims are rising, the strict design is protecting the promise but may need a better user experience.

Common Confusions

Confusion: CAP labels an entire database forever

Why it is tempting:

People say "this database is AP" or "that database is CP" because labels are quick.

Better model:

Ask about one operation and one promise. The same product can make different choices for different state.

Confusion: Availability means the website is reachable

Why it is tempting:

In everyday operations, "available" often means users can load the page.

Better model:

CAP availability is stricter. The specific request to a non-failing node must receive a successful response. "Try later" may be honest and useful, but it is not a successful claim of @luna.

Confusion: Reconciliation makes every local accept safe

Why it is tempting:

Many systems repair data after replication catches up.

Better model:

Reconciliation is safe only when the domain has a real merge or compensation story. It can merge cart additions. It cannot make two people the sole owner of the same username.

Design Review

Choose one operation:

claim username
add item to cart
change password
publish feature flag
edit profile bio
increment notification count

Fill in:

state being changed:
user-visible promise:
two isolated requests that could conflict:
can both accepted outcomes remain true?
partition-time response:
normal-day confirmation rule:
wording the API or UI may safely use:
signal that shows the rule is under pressure:

Model answer for edit profile bio:

state being changed:
  user's profile bio

user-visible promise:
  the latest saved bio should eventually appear

two isolated requests:
  phone saves "Building tools"
  laptop saves "Distributed systems engineer"

can both accepted outcomes remain true?
  not as one final string, but the product can keep both versions or use a conflict UI

partition-time response:
  accept locally if the product is willing to show a later conflict

normal-day confirmation rule:
  local save may be okay if the response means "saved on this device"

wording:
  "Saved locally" or "Syncing" is safer than "Visible everywhere"

signal:
  unresolved profile conflicts and age of unsynced edits

Resources

• [ARTICLE] CAP Twelve Years Later: How the Rules Have Changed - Focus: Eric Brewer's clarification of CAP as an application-level, partition-time trade-off.
• [PAPER] Consistency Tradeoffs in Modern Distributed Database System Design - Focus: The PACELC model and the normal-case latency versus consistency trade-off.
• [BOOK] Designing Data-Intensive Applications - Focus: Replication, linearizability, fault tolerance, and user-visible guarantees.

Key Takeaways

• CAP is an operation-level question about what happens when communication between replicas is broken.
• A local copy can be healthy and still lack the evidence needed to make a global promise.
• PACELC keeps the normal day visible: stronger confirmation often costs latency even without a partition.
• Reconciliation is safe only when the domain has a real merge or compensation rule.
• Response wording should match evidence: pending, saved locally, confirmed, and visible everywhere are different promises.