Day 023: Saga Pattern - Distributed Transactions Without Transactions (Saga pattern fundamentals)

Day 023: Saga Pattern - Distributed Transactions Without Transactions

Topic: Saga pattern fundamentals

💡 Today's "Aha!" Moment

The insight: Distributed transactions (2PC) are dead in microservices—too slow, lock resources, kill availability. Sagas provide eventual consistency through compensating transactions instead of locking.

Why this matters:

Traditional distributed transaction (2PC - Two-Phase Commit):

Coordinator: "Everyone ready to commit?"
Services: "Yes!" (LOCKED, waiting...)
Coordinator: "Commit!" or "Rollback!"
Services: Finally unlock

Problem: If coordinator crashes, services LOCKED FOREVER

2PC sacrifices availability for consistency (violates CAP theorem in distributed systems). One slow service blocks everyone.

Saga pattern: Chain of local transactions, each with compensating action:

Order Saga:
1. Reserve inventory → (Compensate: Release inventory)
2. Charge payment → (Compensate: Refund payment)
3. Ship order → (Compensate: Cancel shipment)

If step fails: Execute compensations in reverse order

Two approaches:

1. Choreography (event-driven, decentralized):

🌟 Why This Matters

The Saga pattern enables reliable distributed transactions in microservices architectures. Companies like Amazon, Uber, and Netflix rely on this pattern to coordinate complex business processes across hundreds of services while maintaining high availability.

Real-world impact:

In traditional monolithic systems, database transactions provide ACID guarantees across the entire application. When you split into microservices, each service owns its data—you can't use a single database transaction spanning multiple services. Two-Phase Commit (2PC) theoretically solves this but fails in practice due to:

Sagas provide a pragmatic alternative: accept eventual consistency, design for compensation. This mindset shift enables:

Modern e-commerce platforms process millions of orders daily using sagas—orders that would be impossible with 2PC locking.

🎯 Daily Objective

Master the Saga pattern: learn how to coordinate long-running business transactions across multiple microservices without distributed transactions. Understand choreography vs orchestration and implement both approaches.

📚 Topics Covered

The Saga Pattern fundamentals:

Sagas solve the distributed transaction problem by decomposing long-running business transactions into a sequence of local transactions. Each local transaction updates the database and publishes an event or message to trigger the next step. If a step fails, the saga executes compensating transactions to undo the changes made by preceding steps.

Key characteristics of sagas:

Two implementation patterns:

1. Choreography (Event-Driven Coordination):

Each service listens for events, performs its local transaction, and publishes new events. No central coordinator exists—services react to events autonomously.

Example flow:

OrderService: Creates order → publishes OrderCreated
InventoryService: Hears event → reserves stock → publishes InventoryReserved
PaymentService: Hears event → charges card → publishes PaymentCharged
ShippingService: Hears event → ships → publishes OrderShipped

If PaymentFailed: publishes PaymentFailed
InventoryService: Hears PaymentFailed → releases stock (compensates)

Pros: No single point of failure, services loosely coupled
Cons: Hard to trace flow, implicit dependencies

2. Orchestration (central coordinator):

SagaOrchestrator:
  1. Call InventoryService.reserve()
  2. If success: Call PaymentService.charge()
  3. If success: Call ShippingService.ship()
  4. If any fails: Execute compensations backward

Orchestrator knows full saga, handles rollback

Pros: Clear flow, easy to trace, centralized logic
Cons: Single point of failure (mitigated by making orchestrator reliable)

The pattern: Long-running transactions via local transactions + compensations

Common misconceptions:

❌ Myth: "Sagas provide ACID transactions"
✅ Truth: Sagas provide BASE (Basically Available, Soft state, Eventual consistency). Intermediate states visible (e.g., inventory reserved but payment pending).

❌ Myth: "Compensations undo everything"
✅ Truth: Some actions can't be undone (email sent, API called). Compensate = mitigate effects (send apology email, call cancellation API).

❌ Myth: "Always use choreography"
✅ Truth: Orchestration better for complex flows with many steps. Choreography better for simple flows with few participants.

Real-world examples:

  1. Uber ride booking:

  2. Reserve driver → Charge rider → Notify driver → Start trip

  3. If charge fails: Release driver (compensation)

  4. Flight booking:

  5. Reserve seat → Charge card → Issue ticket

  6. If card declined: Release seat (automatic compensation)

  7. E-commerce order:

  8. Reserve inventory → Process payment → Create shipment

  9. If shipment fails (out of stock): Refund payment + release inventory

  10. AWS Step Functions: Orchestration service for sagas (visual workflows, automatic retries, compensations)

  11. Netflix Conductor: Orchestration engine for microservices workflows

Meta-insight:

Sagas embrace reality of distributed systems: You can't prevent failures, only handle them gracefully. Instead of locking everything (2PC), progress optimistically and compensate if needed.

This mirrors real life: Hotels overbook (optimistic), then compensate with upgrades if needed. Banks process checks immediately, reverse if bounced (compensating transaction).

The trade-off: Availability > Strong Consistency. Better to complete 99% of orders with occasional visible inconsistency than block everything for perfect consistency.

📖 Detailed Curriculum (Optimized for 60 min)

  1. Video Introduction (15 min) ⭐ START HERE

  2. Chris Richardson: "Managing Data in Microservices"

  3. Video Link

  4. Focus on Saga pattern section (skip CQRS if needed)

  5. Core Reading (15 min)

  6. Microsoft: "Saga Pattern"

  7. Article
  8. Chris Richardson: "Pattern: Saga"
  9. Microservices.io

  10. Focus: Both choreography and orchestration approaches

  11. Quick Synthesis (5 min)

  12. Draw: Choreography vs Orchestration comparison
  13. Write: "When to use each approach"
  14. List: 3 compensating transaction examples

📑 Resources

🎯 Core Resources (Use Today)

⭐ Bonus Resources (If Extra Time)

✍️ Practical Activities (25 min total)

1. Implement Both Saga Patterns (18 min)

Scenario: E-commerce order placement

Services involved:

A. Choreography-Based Saga (8 min)

// Event-driven coordination (no central coordinator)

// ORDER SERVICE
class OrderService {
    createOrder(orderId, items, payment) {
        // Save order
        database.save(Order { id: orderId, status: "pending" })

        // Publish event
        eventBus.publish(OrderCreated {
            orderId, items, payment, userId
        })
    }

    onPaymentSucceeded(event) {
        database.update(event.orderId, { status: "paid" })
        // Next step happens automatically via event
    }

    onPaymentFailed(event) {
        // Compensate: cancel order
        database.update(event.orderId, { status: "cancelled" })
        eventBus.publish(OrderCancelled { orderId: event.orderId })
    }
}

// PAYMENT SERVICE
class PaymentService {
    onOrderCreated(event) {
        try {
            // Attempt payment
            paymentId = processPayment(event.payment)
            database.save(Payment {
                id: paymentId,
                orderId: event.orderId,
                status: "succeeded"
            })

            // Publish success event
            eventBus.publish(PaymentSucceeded {
                orderId: event.orderId,
                paymentId
            })
        } catch (error) {
            // Publish failure event
            eventBus.publish(PaymentFailed {
                orderId: event.orderId,
                reason: error
            })
        }
    }

    onOrderCancelled(event) {
        // Compensate: refund payment
        payment = database.find(event.orderId)
        if (payment && payment.status == "succeeded") {
            refund(payment.id)
            database.update(payment.id, { status: "refunded" })
        }
    }
}

// INVENTORY SERVICE
class InventoryService {
    onPaymentSucceeded(event) {
        try {
            // Reserve items
            for item in event.items {
                reserveItem(item.id, item.quantity)
            }

            eventBus.publish(InventoryReserved {
                orderId: event.orderId
            })
        } catch (error) {
            // Can't reserve - trigger compensation
            eventBus.publish(InventoryReservationFailed {
                orderId: event.orderId
            })
        }
    }

    onOrderCancelled(event) {
        // Compensate: release reserved items
        releaseReservation(event.orderId)
    }
}

B. Orchestration-Based Saga (10 min)

// Central coordinator manages the workflow

class OrderSagaOrchestrator {
    eventStore

    async createOrder(orderId, items, payment, userId) {
        saga = new SagaInstance(orderId)

        try {
            // Step 1: Create Order
            saga.addStep({
                action: () => orderService.createOrder(orderId, items),
                compensation: () => orderService.cancelOrder(orderId)
            })
            await saga.executeStep(1)

            // Step 2: Process Payment
            saga.addStep({
                action: () => paymentService.processPayment(payment),
                compensation: () => paymentService.refund(orderId)
            })
            paymentId = await saga.executeStep(2)

            // Step 3: Reserve Inventory
            saga.addStep({
                action: () => inventoryService.reserve(items),
                compensation: () => inventoryService.release(orderId)
            })
            await saga.executeStep(3)

            // Step 4: Schedule Delivery
            saga.addStep({
                action: () => deliveryService.schedule(orderId),
                compensation: () => deliveryService.cancel(orderId)
            })
            await saga.executeStep(4)

            // Success!
            saga.complete()
            return { success: true, orderId }

        } catch (error) {
            // Failure - run compensating transactions
            await saga.compensate()
            return { success: false, reason: error }
        }
    }
}

class SagaInstance {
    orderId
    steps = []
    currentStep = 0

    addStep(step) {
        steps.push(step)
    }

    async executeStep(stepNumber) {
        currentStep = stepNumber
        step = steps[stepNumber - 1]

        // Save saga state
        eventStore.append(SagaStepStarted {
            sagaId: orderId,
            step: stepNumber
        })

        // Execute action
        result = await step.action()

        // Save completion
        eventStore.append(SagaStepCompleted {
            sagaId: orderId,
            step: stepNumber
        })

        return result
    }

    async compensate() {
        // Run compensations in reverse order
        for (i = currentStep - 1; i >= 0; i--) {
            step = steps[i]
            await step.compensation()

            eventStore.append(SagaStepCompensated {
                sagaId: orderId,
                step: i + 1
            })
        }
    }

    complete() {
        eventStore.append(SagaCompleted { sagaId: orderId })
    }
}

2. Comparison Diagram (5 min)

Draw both patterns side-by-side showing the flow:

Choreography:

OrderService → Event → PaymentService → Event → InventoryService
     ↓                      ↓                          ↓
   Events               Events                     Events

Orchestration:

        Orchestrator
            ↓ ↑
    Order → Payment → Inventory → Delivery
            ↑ (on failure, compensate)

3. Quick Analysis (2 min)

Fill in:
| Aspect | Choreography | Orchestration |
|--------|--------------|---------------|
| Coupling | Loose | Tighter |
| Visibility | Distributed | Central |
| Complexity | Per service | In orchestrator |
| Best for | Simple flows | Complex flows |

🎨 Creativity - Quick Mental Reset (5 min)

Quick Exercise: "The Dance vs The Conductor"

Draw two scenarios:

  1. Dance (Choreography): Dancers responding to each other's moves
  2. Orchestra (Orchestration): Conductor directing musicians

Label each with:

Purpose: Visualize the core difference between saga approaches.

Take 3 min to draw, 2 min to reflect on which pattern fits which scenarios

🔗 Connections to Previous Learning

From This Week:

From Month 1:

Building Forward:

✅ Daily Deliverables (Must Complete)

⭐ Bonus (If Extra Time)

🎯 Success Criteria

By the end of today, you should be able to:

  1. ✅ Explain what sagas are and why they're needed
  2. ✅ Implement choreography-based saga
  3. ✅ Implement orchestration-based saga
  4. ✅ Write compensating transactions
  5. ✅ Choose between choreography and orchestration
  6. ✅ Handle saga failures gracefully

⏰ Total Estimated Time (OPTIMIZED)

Note: Pseudocode is perfect for demonstrating saga patterns. Focus on the coordination logic, not implementation details!

📝 Today's Big Ideas

  1. No Distributed Transactions: Sagas maintain consistency without 2PC
  2. Compensating Transactions: Undo operations when saga fails
  3. Eventual Consistency: Saga completes over time, not atomically
  4. Two Approaches: Choreography (events) vs Orchestration (coordinator)
  5. Production Pattern: Used by Uber, Netflix, Amazon for critical flows

💡 Saga in the Real World

🚀 Tomorrow's Preview

Stream Processing: Continuous Data Flows

You'll learn how to process continuous streams of events in real-time. Apache Kafka, stream processing patterns, and building reactive systems!

"Sagas are a mechanism for maintaining consistency in a distributed system without requiring distributed transactions." - Chris Richardson

You're building microservices-ready patterns! 💪

← Back to Learning