The Saga Pattern: Choreography or Orchestration?

Version note: The code in this post was built and run with C# 14 and .NET 10.

Guess First

Bookstore's checkout is a single method today: create the order, reserve stock, take the money from the payment provider, create a shipment through the carrier's API (Application Programming Interface). Each line is really a call to another system; the payment provider and the carrier are outside Bookstore already.

namespace Bookstore.Sagas;

// Each line is really a call to another system's API
public sealed class CheckoutService(
    OrderingService ordering, InventoryService inventory, PaymentService payments, ShippingService shipping)
{
    public string Checkout(OrderId id, IReadOnlyList<OrderedBook> books, decimal total)
    {
        ordering.Place(id);
        if (!inventory.TryReserve(id, books, out string? missing))
        {
            ordering.Cancel(id, "OutOfStock");
            return $"out of stock: {missing}";
        }
        if (!payments.TryCapture(id, total))
        {
            inventory.Release(id);
            ordering.Cancel(id, "PaymentFailed");
            return "payment failed";
        }
        ordering.MarkPaid(id);
        shipping.CreateShipment(id);
        return "confirmed";
    }
}

The code looks like it thought about failures: no stock means cancel, a declined payment means release the stock and cancel. But what happens if the carrier's API times out? Or if the process restarts right after TryCapture: who will know what to undo, and in which order?

Saga: A Sequence of Local Transactions Instead of One Transaction

I tried it with the carrier API turned off:

1) Checkout as one method; the carrier API times out
  [Ordering ] order -> Placed
  [Inventory] reserved 2 x Dune
  [Payments ] captured 60.00 EUR
  [Ordering ] order -> Paid
  checkout threw: Carrier API timed out
  after the crash: stock [Dune 1, Middlemarch 5], captured 60.00 EUR, order Paid, shipments 0

The money was taken, two copies of Dune are reserved, the order looks "paid", and there is no shipment. The moment the exception left the method, the knowledge of "what have I done so far and what was next" vanished too; it lived only on the call stack. This code is a hidden saga: a piece of work that changes several systems and can take a while, yet it neither stores its state nor knows what to do if it stops halfway.

The concept comes from a database paper: "Sagas" by Hector Garcia-Molina and Kenneth Salem, published in 1987 at SIGMOD (the Association for Computing Machinery's Special Interest Group on Management of Data) conference. The problem was LLTs (long-lived transactions) that ran for hours or days: they held locks for a long time, kept other work waiting and raised the deadlock and abort rates. The proposal was to split such work into sub-transactions that commit on their own (T1, T2, …, Tn) and to pair each with a compensating transaction (C1, C2, …) that undoes it semantically. The system guarantees one of two outcomes: either all of T1…Tn run, or the ones that ran up to some point j are compensated in reverse order (T1…Tj, Cj…C1). Two details of the paper still hold today. First, compensation does not restore the database to its earlier state; in the airline seat example, the transaction that cancels a reservation does not write the old seat count back, because others may have booked in between, it decrements by one. Second, other transactions can see a saga's partial results; isolation is given up on purpose. The paper also describes a coordinator that writes the saga's state to a log (the SEC, saga execution coordinator): after a crash the log is read and an unfinished saga is either rolled back or continued from where it stopped (forward recovery).

The microservices world took this idea out of a single database and put it between services. Microsoft Learn's Saga pattern page summarizes it like this: each step is an atomic local transaction in its own service's database, triggers the next step through an event or message, and if a step fails, the compensating transactions of the earlier steps run. Even though Bookstore is a monolith with one database today, it needs a saga, because the payment provider and the carrier will never join that database's transaction.

Compensable, Pivot and Retryable Steps

Not all steps are equal. Microsoft Learn's page and Chris Richardson's Microservices Patterns (2018) book split them into three groups. A compensable step can be undone by an operation with the opposite effect. The pivot step is the saga's point of no return: once it succeeds, the saga is no longer rolled back, it moves forward. A retryable step comes after the pivot and is assumed to succeed eventually, so it must be idempotent. Bookstore's order saga looks like this:

Where the pivot sits is a business decision, not a technical one. Refunding is possible, but a refund means two lines on the customer's statement, a fee to the payment provider and a support ticket; Bookstore treats the capture as irreversible by saying "if the payment was taken, we ship the book". That decision has two effects on the design. Steps that are likely to fail and cheap to undo should come before the pivot: if there is no stock, we want to know before taking the money. Every step after the pivot must replace the question "what if it fails?" with "how do we retry until it succeeds?"; if the carrier API is down for an hour, the order is not cancelled, the shipment is created an hour later. That was the real bug in the opening code: it threw a post-pivot failure up the stack as if it were a reason to roll back, and wrote the state nowhere. On the payment side there is a well-known way to push the pivot later: first only authorize the card (its compensation is voiding the authorization) and capture when the shipment is created. Then the authorization becomes a compensable step and the capture moves to the end.

Compensation Is Not Rollback

Garcia-Molina's seat example is even more visible once the steps live in separate services. Compensation is not an "undo" but a new operation that means something to the business. A sent email cannot be unsent; you send a second "your order was cancelled" email. A refund is not the deletion of a capture but a separate money movement. Releasing a stock reservation does not write Reserved back to its old value, it adds the reserved quantity back, because other orders may have reserved in between.

That is why compensation needs three properties. It must be idempotent: compensation messages are delivered at least once too and can arrive twice. In the example, InventoryService.Release does nothing if there is no reservation record. It must undo only what exists: calling "refund" in a saga that never reached the payment step means trying to refund an amount that was never captured. And it must not depend on order: since message order is not guaranteed, a "release the reservation" command can sometimes arrive before the reservation itself. Silently skipping it because there is nothing to release locks the stock forever when the reservation arrives a few milliseconds later; the robust fix is to remember "this order was cancelled" and reject the late reservation.

Choreography: Coordination Through Events

The first way to run a saga is choreography: there is no central coordinator, each service listens to other services' events, runs its own local transaction and publishes the result as an event again. I built the same flow on a small in-memory bus that delivers messages from a queue one by one:

public static class Choreography
{
    public static void Wire(InMemoryBus bus, OrderingService ordering, InventoryService inventory,
        PaymentService payments, ShippingService shipping)
    {
        // Ordering: derives the order's status from other contexts' events
        bus.Subscribe<StockReserved>(e => ordering.MarkPaymentPending(e.OrderId));
        bus.Subscribe<PaymentCaptured>(e => ordering.MarkPaid(e.OrderId));
        bus.Subscribe<PaymentFailed>(e => ordering.Cancel(e.OrderId, "PaymentFailed"));
        bus.Subscribe<OutOfStock>(e => ordering.Cancel(e.OrderId, "OutOfStock"));

        // Inventory: reserves stock on an order, releases it if the payment fails (compensation)
        bus.Subscribe<OrderPlaced>(e => bus.Publish(inventory.TryReserve(e.OrderId, e.Books, out string? missing)
            ? new StockReserved(e.OrderId)
            : new OutOfStock(e.OrderId, missing!)));
        bus.Subscribe<PaymentFailed>(e => inventory.Release(e.OrderId));

        // Payments: learns the amount from OrderPlaced, captures when Inventory's StockReserved arrives
        Dictionary<OrderId, decimal> amounts = [];
        bus.Subscribe<OrderPlaced>(e => amounts[e.OrderId] = e.Total);
        bus.Subscribe<StockReserved>(e => bus.Publish(payments.TryCapture(e.OrderId, amounts[e.OrderId])
            ? new PaymentCaptured(e.OrderId, amounts[e.OrderId])
            : new PaymentFailed(e.OrderId, "card declined")));

        // Shipping: creates the shipment once the payment is taken
        bus.Subscribe<PaymentCaptured>(e => bus.Publish(new ShipmentCreated(e.OrderId, shipping.CreateShipment(e.OrderId))));
    }
}

Choreography: 3 x Middlemarch, 135.00 EUR
  [Ordering ] order -> Placed
  OrderPlaced
  [Inventory] reserved 3 x Middlemarch
  StockReserved
  [Ordering ] order -> PaymentPending
  [Payments ] declined 135.00 EUR
  PaymentFailed
  [Ordering ] order -> Cancelled (PaymentFailed)
  [Inventory] released 3 x Middlemarch
  result: order Cancelled, stock [Dune 3, Middlemarch 5], captured 0.00 EUR

The payment provider declines anything above 100 EUR; the 135 EUR order was cancelled and the three copies of Middlemarch were released. The code is short and needs no extra component. Read it carefully, though, and the cost shows. Payments has to know Inventory's StockReserved event, and it keeps the amount from OrderPlaced itself, because the stock event does not carry the payment amount (if it did, Inventory's contract would depend on a payment-related field). The flow is written nowhere; the answer to "under which condition is the payment taken?" is spread over four different subscriptions. Adding a step, say a fraud check before the payment, means changing which event Payments listens to; adding a step requires touching its neighbour.

Orchestration: A Central Coordinator

The second way is orchestration: a single component that knows the flow, the orchestrator, tells participants what to do with commands and receives their answers as events. In Hohpe and Woolf's EIP (Enterprise Integration Patterns, 2003) catalog this is the Process Manager. The same saga, this time with an OrderSaga living inside Ordering:

public enum SagaStep { ReservingStock, CapturingPayment, CreatingShipment, ReleasingStock, Completed, Cancelled }

// Orchestrator: the whole flow in one place; participants only know their own commands
public sealed class OrderSaga(InMemoryBus bus, OrderingService ordering)
{
    private readonly Dictionary<OrderId, (SagaStep Step, decimal Total)> _sagas = []; // moves to the database in post 09

    public void Handle(object message)
    {
        switch (message)
        {
            case OrderPlaced e:
                _sagas[e.OrderId] = (SagaStep.ReservingStock, e.Total);
                Log.Write("OrderSaga", "started -> ReservingStock");
                bus.Publish(new ReserveStock(e.OrderId, e.Books));
                break;
            case StockReserved e:
                MoveTo(e.OrderId, SagaStep.CapturingPayment);
                ordering.MarkPaymentPending(e.OrderId);
                bus.Publish(new CapturePayment(e.OrderId, _sagas[e.OrderId].Total));
                break;
            case OutOfStock e:
                MoveTo(e.OrderId, SagaStep.Cancelled);
                ordering.Cancel(e.OrderId, "OutOfStock");
                break;
            case PaymentCaptured e:
                MoveTo(e.OrderId, SagaStep.CreatingShipment);
                ordering.MarkPaid(e.OrderId);
                bus.Publish(new CreateShipment(e.OrderId));
                break;
            case PaymentFailed e:
                MoveTo(e.OrderId, SagaStep.ReleasingStock);
                bus.Publish(new ReleaseStock(e.OrderId));
                break;
            case StockReleased e:
                MoveTo(e.OrderId, SagaStep.Cancelled);
                ordering.Cancel(e.OrderId, "PaymentFailed");
                break;
            case ShipmentCreated e:
                MoveTo(e.OrderId, SagaStep.Completed);
                break;
        }
    }

    private void MoveTo(OrderId id, SagaStep next)
    {
        Log.Write("OrderSaga", $"{_sagas[id].Step} -> {next}");
        _sagas[id] = _sagas[id] with { Step = next };
    }
}

Orchestration: 3 x Middlemarch, 135.00 EUR
  [Ordering ] order -> Placed
  OrderPlaced
  [OrderSaga] started -> ReservingStock
  ReserveStock
  [Inventory] reserved 3 x Middlemarch
  StockReserved
  [OrderSaga] ReservingStock -> CapturingPayment
  [Ordering ] order -> PaymentPending
  CapturePayment
  [Payments ] declined 135.00 EUR
  PaymentFailed
  [OrderSaga] CapturingPayment -> ReleasingStock
  ReleaseStock
  [Inventory] released 3 x Middlemarch
  StockReleased
  [OrderSaga] ReleasingStock -> Cancelled
  [Ordering ] order -> Cancelled (PaymentFailed)
  result: order Cancelled, stock [Dune 3, Middlemarch 5], captured 0.00 EUR

Same result, more messages: every step is a command and a reply. In return, the flow reads in a single switch, the step the saga is in at any moment (CapturingPayment, ReleasingStock) exists as a value, and the participants know nothing about each other. Inventory no longer knows Payments, and Payments no longer knows Inventory; each only knows its own command and reply. Adding a fraud check means adding a step to OrderSaga. In the example the saga's state lives in a Dictionary; if the process restarts, it is no better than the opening code. Storing the state in the database, timeouts and concurrent messages are the subject of the next post.

Which One When?

Microsoft Learn's summary is similar: choreography suits simple flows with few services; as steps and services grow, orchestration's logic kept in one place wins. A rule that works well in practice: choreography for a flow of two or three steps owned by one team with simple compensation; orchestration for a flow with a pivot, timeouts, more than one compensation path, or one where the business asks "which step is this order in right now?". The two are also used together: Bookstore's order saga becomes an orchestrator inside Ordering, while reactions that do not affect the flow, such as Notification, still hang off events like OrderPlaced through choreography. There is one rule against the orchestrator getting bloated: it should know only the order of steps and the decisions; business rules such as "how is stock reserved" or "how is the shipping fee calculated" stay in the participants' own models.

No Isolation: Saga Anomalies and Countermeasures

The "I" of ACID (Atomicity, Consistency, Isolation, Durability) is missing in a saga: every step commits its own transaction and its results are visible to others before the whole saga ends. Microsoft Learn's page lists three anomalies that follow. Lost update: one saga overwrites another saga's change without seeing it. Dirty read: a saga reads a change that is not finished yet and may be undone by compensation; telling another customer "out of stock" because of the last Middlemarch reserved by an order whose payment will be declined a few seconds later is the typical Bookstore example. Fuzzy (nonrepeatable) read: two steps of the same saga read different values because something changed in between.

The same page lists countermeasures. Semantic lock: the compensable step puts an application-level marker on the record saying "work in progress"; in Bookstore the PaymentPending status is exactly that. Commutative updates: design updates so that order does not matter; for stock, "add/subtract this much" instead of "write this value". Pessimistic view: reorder the saga so that risky updates move into retryable steps. Reread values: check that a value has not changed before updating it, a form of optimistic concurrency; like verifying that the price is still valid before capturing the payment. Version files: keep a log of operations on a record and put out-of-order ones into the right sequence. Risk-based concurrency: choose the mechanism by value; sagas for low-risk work, something stricter for high-risk work.

In Bookstore, the semantic lock is a small guard:

public void CancelByCustomer(OrderId id)
{
    // Semantic lock: the customer cannot cancel while the saga is in the payment step
    if (Orders[id] is OrderStatus.PaymentPending)
        throw new DomainException("order.payment-pending", "Payment is in progress; the order cannot be cancelled right now.");
    Cancel(id, "CustomerRequest");
}

Semantic lock: the customer cancels while payment is in progress
  [Ordering ] order -> Placed
  [Ordering ] order -> PaymentPending
  order.payment-pending: Payment is in progress; the order cannot be cancelled right now.

Without the semantic lock, the customer cancels the order, Inventory releases the stock, and a few hundred milliseconds later PaymentCaptured arrives from the payment provider: money taken for a cancelled order. The cost of a semantic lock shows up in the user experience: instead of saying "you cannot cancel right now", a kinder solution records the cancellation request and handles it with a refund once the saga passes the pivot; but that adds one more path to the saga. The lock also needs an owner and a lifetime: if the saga crashes, an order stuck in PaymentPending must not stay locked forever. Solving that with timeouts is the subject of the next post.

Common Pitfalls

Presenting a saga as an ACID transaction

Telling the user "your order is confirmed" before the pivot is a lie when the saga is rolled back with compensation a few seconds later. While the saga runs, the state shown to the user is an intermediate one (Placed, PaymentPending); the user interface should present it as "processing" and announce the outcome through polling, email or push.

Throwing after the pivot

The opening code did exactly this. A failure in a post-pivot step is a reason to retry, not to roll back. If the carrier API is down, the saga should stay in a "waiting for shipment" step and retry; when retries run out, a human should step in.

Not testing compensation as much as the happy path

Compensation code runs rarely, so it is the least tested code, and it runs exactly when the system is already in trouble. For every compensable step, test "this step succeeded, the next one was rejected", the compensation arriving twice, and the compensation arriving before the original operation.

Cycles in choreography

Once Inventory listens to PaymentFailed, Payments to StockReleased, and Ordering to both, events can turn into a loop that keeps triggering itself. Microsoft Learn also lists the risk of cyclic dependencies between participants among choreography's drawbacks. If there is no diagram of the event flow, drawing one is the cheapest way to spot the loop.

A saga for everything

Building a saga for an operation that finishes inside one database, one aggregate or one bounded context means throwing away free ACID guarantees and taking on eventual consistency, compensation and anomaly handling instead.

When Not to Use It

If all steps live in the same database, use a local transaction. In the "Domain Events" post of the DDD (Domain-Driven Design) series we put stock reservation in the same transaction as the order; as long as Bookstore stays on one database, that is both simpler and more correct than a two-step saga. Steps whose compensation cannot be defined do not fit a saga either: Microsoft Learn says the pattern may not be suitable when transactions are tightly coupled, when compensating transactions occur in earlier participants, or when there are cyclic dependencies. And if the business truly needs all-or-nothing (a money transfer between two accounts, say), redrawing the boundary so that both sides live in one database is often a better decision than accepting eventual consistency.

A Small Experiment

★☆☆ Place an order that runs out of stock (4 x Dune) in the choreography example. Which events are published, does Payments ever see this order? What is left in Payments' amounts dictionary, and why is that a small memory leak?

★★☆ Add a CheckFraud step before the pivot in the orchestration example. How many files changed? Which subscriptions would you have to change to add the same step to the choreography?

★★★ Arrange an order in which the ReleaseStock command is delivered before ReserveStock (put the two messages on the bus in reverse order). What does InventoryService.Release do, and what happens to the stock when the reservation arrives afterwards? Change Release so that it remembers cancelled orders.

The OrderSaga in the example makes every decision correctly but keeps its state in memory; it is as fragile as the opening code. In the next post we turn it into a state machine persisted with Entity Framework Core (EF Core), and look at matching messages by correlation id, optimistic concurrency, timeouts, and the case where compensation itself fails; we will also see the same saga in MassTransit and Wolverine.