Outbox, Inbox, and Dual-Write Avoidance

By the end of this lesson, you will be able to...

• Trace why a database update plus an event publish can split into two different truths.

• Explain how an outbox, a relay, and an inbox reduce lost events and duplicate side effects.

• Review the trade-off between stronger application safety and more operational moving parts.

Idea in one sentence: An outbox keeps the local state change and the intent to publish in one database transaction, while an inbox helps each consumer make repeated delivery safe.

Core Insight

An order service confirms payment for order O-812.

Two things must happen:

1. Mark the payment as captured in the order database.
2. Publish PaymentCaptured so fulfillment can reserve stock.

The product promise sounds simple:

If the payment is captured, fulfillment should eventually hear about it.

The naive implementation is also simple:

BEGIN
  update orders set payment_status = 'captured'
COMMIT

publish PaymentCaptured(O-812)

But the database and the broker are two different systems. The database commit can succeed while the publish fails. The publish can succeed while the application crashes before it records that fact. A retry can publish the same event twice.

This is the dual-write problem.

The hard part is not that events are unreliable. The hard part is that one business decision is being written to two places that do not share one atomic commit.

Outbox and inbox patterns do not make a distributed transaction appear by magic. They change where the system keeps evidence. The producer records the business change and the event-to-publish together. Each consumer records which message it already handled before it performs an unsafe side effect again.

That turns a vague hope into an inspectable mechanism.

The Naive Dual Write

Start with the most direct implementation.

The order service owns the fact that a payment was captured. Fulfillment should not decide that fact. Fulfillment only reacts to it.

Order service
  -> write local payment state
  -> publish PaymentCaptured

Fulfillment service
  -> receive PaymentCaptured
  -> reserve stock

This looks reasonable because each step is simple.

Now put failures between the steps.

The uncomfortable part is the gap between step 1 and step 2.

If the database commit succeeds and the publish fails, the order is paid but fulfillment never hears about it. If the service retries the whole command blindly, it might publish twice. If the publish happens before the database commit, a consumer can react to an event for a fact that later rolls back.

There are three common failure shapes:

• Commit without publish: local truth changes, but no event leaves.
• Publish without durable local truth: consumers react to a fact that may not exist.
• Duplicate publish or delivery: consumers see the same fact more than once.

The broker can help with delivery. It cannot know the producer's database transaction. The database can protect local rows. It cannot know whether every subscriber handled the event.

So the application needs a boundary pattern.

The Outbox Mechanism

The outbox pattern adds one table inside the producer's database.

It usually contains messages that still need to be published:

Now the command handler does not write to the database and broker in one breath. It writes the local state and the outbox row in the same database transaction.

BEGIN
  update orders
    set payment_status = 'captured'
    where order_id = 'O-812';

  insert into outbox_messages
    (message_id, aggregate_id, event_type, payload, created_at)
    values
    ('msg-441', 'O-812', 'PaymentCaptured', '{...}', now());
COMMIT

After the commit, a relay process reads unpublished outbox rows and publishes them to the broker.

outbox row -> relay -> broker topic -> consumers

The important change is the atomic boundary.

The database transaction now says:

Either the payment state and the intent to publish are both recorded,
or neither is recorded.

That does not guarantee that the broker already has the message. It guarantees that the system has durable evidence that a message must be published.

If the order service crashes after the commit, the outbox row is still there. When the relay restarts, it can publish it.

If the relay publishes the message but crashes before marking published_at, it may publish again. That is why outbox reduces lost events but does not remove duplicate delivery.

A Worked Failure Trace

Trace the same payment with and without an outbox.

Without an outbox

The output is inconsistent:

Order says paid.
Fulfillment never receives PaymentCaptured.

The system has no durable "publish still required" record unless someone infers it later from payment state.

With an outbox

Now the output is different:

Order says paid.
Outbox says PaymentCaptured must be published.
Fulfillment eventually receives it.

But there is still a duplicate path.

If the relay publishes at 10:06 and crashes before 10:07, it may publish msg-441 again after restart. That is usually acceptable only if consumers treat msg-441 as the same message, not as a new business fact.

This is where the inbox pattern enters.

The Inbox Mechanism

The outbox protects the producer side of the boundary.

The inbox protects the consumer side.

Fulfillment receives PaymentCaptured(msg-441, order_id=O-812).

It wants to reserve stock exactly once for that payment event. The broker may deliver the message more than once. The relay may publish it more than once. A consumer may crash after reserving stock but before acknowledging the message.

An inbox table records which messages this consumer has already processed.

A common consumer flow looks like this:

receive PaymentCaptured(msg-441)

BEGIN
  if inbox already has (fulfillment-reserver, msg-441):
    skip side effect
  else:
    reserve stock for order O-812
    insert inbox row (fulfillment-reserver, msg-441)
COMMIT

acknowledge message

The exact implementation varies. Sometimes the inbox row is inserted first with a unique constraint. Sometimes the side effect is a local state change in the same database transaction. Sometimes an external side effect needs its own idempotency key.

The principle is stable:

A retry should see the same message id and avoid repeating unsafe work.

This is not only a broker concern. It is application state.

The consumer knows whether reserving stock twice is dangerous. The broker does not.

What This Pattern Guarantees

Outbox plus inbox gives a useful application-level promise:

When the producer commits a local fact, the intent to publish that fact is durable.
When a consumer sees the same message again, it has a local way to avoid repeating the same side effect.

That is not the same as saying:

Every consumer definitely succeeded exactly once.

The pattern usually supports at-least-once delivery with idempotent handling.

That phrase means:

• A committed event should eventually be delivered if the relay and broker recover.
• The same event may be delivered more than once.
• Consumers must make repeated delivery safe for their own side effects.

This matches the earlier lessons on delivery semantics and idempotency boundaries. The producer owns the fact and the outbox record. The relay owns publishing attempts. Each consumer owns its own deduplication and side-effect safety.

Trade-offs and Limits

The outbox pattern improves recovery across the database-broker boundary.

It also adds work:

• You need an outbox table, cleanup policy, relay, monitoring, and retry behavior.
• You need to detect rows that are stuck unpublished.
• You need stable message ids and clear event contracts.
• You need a decision about ordering, especially if multiple outbox rows belong to the same aggregate.

The inbox pattern improves duplicate safety.

It also adds cost:

• Each consumer needs storage for processed message ids.
• External side effects may need their own idempotency keys.
• Cleanup must not delete inbox evidence too early if old messages can reappear.
• Deduplication does not fix a wrong business event or a broken schema.

The pattern also has boundaries.

It does not replace a workflow engine. It does not design compensation for a long-running saga. It does not prove that every downstream service completed. It does not remove the need for monitoring lag, relay failures, DLQs, and poison events.

Its main job is narrower and valuable:

Make crossing one local database and one event stream recoverable and reviewable.

Check: A relay publishes PaymentCaptured(msg-441) and then crashes before marking the outbox row as published. What should the system assume after restart?

Think first, then reveal.

Answer: The relay may publish msg-441 again. Consumers must treat msg-441 as the same message and avoid repeating unsafe side effects. The outbox prevents a lost publish intent; it does not remove duplicates.

Review Questions for a Design

When you see an event-driven service that writes local state and publishes an event, ask these questions:

These questions keep the pattern concrete. They stop the review from turning into a vague claim that "we use events, so services are decoupled."

Check: A payment consumer says, "The broker guarantees at-least-once delivery, so we do not need an inbox." What is the flaw?

Think first, then reveal.

Answer: At-least-once delivery allows duplicates. The broker may deliver the same event again after a retry or crash. The consumer still needs an idempotency boundary for work that should not happen twice.

Practice

Review this design.

Billing service:
  1. Charge card through a payment provider.
  2. Update invoices.status = 'paid'.
  3. Publish InvoicePaid(invoice_id).

Email service:
  1. Receive InvoicePaid.
  2. Send receipt email.
  3. Acknowledge message.

Name two failure windows. Then propose the smallest outbox or inbox change that reduces each risk.

Model answer:

• If billing updates invoices.status and crashes before publishing InvoicePaid, the invoice is paid but email never hears about it. Add an outbox row in the same transaction as the invoice update, then let a relay publish from the outbox.
• If the email service sends the receipt and crashes before acknowledging the message, the broker may redeliver and the user may receive two receipts. Add an inbox or idempotency record keyed by message_id and consumer_name, or use a receipt-send operation with a stable idempotency key.

A stronger answer also notices a boundary: charging the external payment provider is not automatically protected by the database outbox. That call may need its own idempotency key with the provider, or the workflow may need a deeper saga or orchestration design.

Resources

• [BOOK] Martin Kleppmann - Designing Data-Intensive Applications
  • Link: https://dataintensive.net/
  • Focus: Use for the broader reliability context around logs, messaging, retries, idempotence, and the difficulty of atomic commits across systems.
• [ARTICLE] Chris Richardson - Transactional Outbox
  • Link: https://microservices.io/patterns/data/transactional-outbox.html
  • Focus: Use for the producer-side pattern: persist the business update and the message to send inside one local transaction.
• [ARTICLE] Chris Richardson - Idempotent Consumer
  • Link: https://microservices.io/patterns/communication-style/idempotent-consumer.html
  • Focus: Use for the consumer-side pattern: record processed message ids so repeated delivery does not repeat unsafe work.

Key Takeaways

• A dual write happens when one business decision must be recorded in two systems, such as a database and a broker, without one shared atomic commit.
• An outbox records the local state change and the publish intent in the same producer database transaction.
• A relay publishes outbox rows later, so a crash after commit can be recovered.
• The relay can still publish duplicates, so consumers need inbox records or other idempotency keys for unsafe side effects.
• Outbox and inbox patterns improve application-level reliability, but they add operational state and do not replace deeper saga, workflow, or broker-internals design.