Implementing a Saga in .NET: Persisted State Machines, Timeouts, and Compensation

Version note: The code in this post was built and run with C# 14 and .NET 10; the saga state is stored with Entity Framework Core (EF Core) 10 and SQLite, and the comparison examples were run with MassTransit 8.5.12 and Wolverine 6.62.0.

Guess First

The orchestrator from the previous post made every decision correctly, but it kept its state here:

    private readonly Dictionary<OrderId, (SagaStep Step, decimal Total)> _sagas = []; // moves to the database in post 09

And it moved to the payment step like this:

            case StockReserved e:
                MoveTo(e.OrderId, SagaStep.CapturingPayment);
                ordering.MarkPaymentPending(e.OrderId);
                bus.Publish(new CapturePayment(e.OrderId, _sagas[e.OrderId].Total));
                break;

Three questions. If the application restarts while the saga is in CapturingPayment, what will the reply from the payment provider be matched against? If the payment provider never answers, how long should the saga wait, and should it then release the stock and cancel the order? If two instances handle PaymentCaptured and a timeout at the same moment, which one wins?

The Process Manager's State: The Saga Table

Hohpe and Woolf's Process Manager and Garcia-Molina's saga execution coordinator say the same thing: the coordinator's state must survive a crash. The first step is one database row per saga instance:

// The saga's persisted state: not the domain model, but the record of a workflow
public sealed class OrderSagaState
{
    public Guid OrderId { get; private set; }            // correlation id and primary key
    public SagaStep Step { get; private set; }
    public decimal Total { get; private set; }
    public DateTime? DeadlineUtc { get; private set; }   // timeout of the active step
    public int Attempts { get; private set; }           // attempt number within the same step
    public int Version { get; private set; }            // optimistic concurrency token
    public DateTime UpdatedUtc { get; private set; }

    public static OrderSagaState Start(OrderPlaced placed, DateTime nowUtc) => new()
    {
        OrderId = placed.OrderId, Step = SagaStep.ReservingStock, Total = placed.Total,
        DeadlineUtc = nowUtc + OrderSagaMachine.StockTimeout, Attempts = 1, UpdatedUtc = nowUtc
    };

    public void Apply(Decision decision, DateTime nowUtc)
    {
        Attempts = decision.Next == Step ? Attempts + 1 : 1;
        Step = decision.Next;
        DeadlineUtc = decision.Timeout is { } t ? nowUtc + t : null;
        UpdatedUtc = nowUtc;
        Version++;
    }
}

        b.Entity<OrderSagaState>(s =>
        {
            s.ToTable("OrderSagas");
            s.HasKey(x => x.OrderId);
            s.Property(x => x.Step).HasConversion<string>();
            s.Property(x => x.Version).IsConcurrencyToken();
            s.HasIndex(x => x.DeadlineUtc); // for the timeout scan
        });

There are four decisions here. OrderId was chosen as the correlation id and made the primary key: an order has exactly one saga, every message carries OrderId, and finding the saga is a primary key read. If the same OrderPlaced arrives twice, the second INSERT is rejected by the primary key, so the start is idempotent by itself. Step is stored as a string, because one day a support engineer will look at this table with SQL (Structured Query Language), and seeing CapturingPayment is far more useful than seeing 1. Version is a concurrency token, incremented on every Apply; we will see shortly why it is needed. DeadlineUtc says until when the active step waits for an answer and is indexed; the timeout scan will run against it.

This row is not the Order aggregate. The order is a concept of the domain itself and lives by the rules we built in the DDD (Domain-Driven Design) series; the saga state answers "which message am I waiting for for this order, and until when", the record of a workflow. Merging the two in one class adds fields to Order that the domain expert has never heard of, such as PaymentTimeoutTokenId. The saga changes the order's status through commands like MarkOrderPaid and CancelOrder; in Bookstore the saga lives inside Ordering, so these are handlers running against the same database.

A Pure State Machine: Deciding with a switch Expression

Separating the deciding code from I/O is the most important step towards a testable saga. The decision is a pure function that takes the current state and the incoming message and produces the new step, the new timeout and the commands to send:

public sealed record Decision(SagaStep Next, TimeSpan? Timeout, IReadOnlyList<IOrderMessage> Commands);

// Pure function: (current state, incoming message) -> decision. No I/O, no database; easy to test.
public static class OrderSagaMachine
{
    public static readonly TimeSpan StockTimeout = TimeSpan.FromMinutes(1);
    public static readonly TimeSpan PaymentTimeout = TimeSpan.FromSeconds(30);
    public static readonly TimeSpan ShipmentTimeout = TimeSpan.FromMinutes(5);
    public const int MaxShipmentAttempts = 3;

    public static Decision? Decide(OrderSagaState saga, IOrderMessage message)
    {
        // Ignore a late timeout that belongs to an earlier step
        if (message is SagaTimedOut timedOut && timedOut.Step != saga.Step)
            return null;

        Guid id = saga.OrderId;
        return (saga.Step, message) switch
        {
            (SagaStep.ReservingStock, StockReserved) =>
                Go(SagaStep.CapturingPayment, PaymentTimeout, new CapturePayment(id, saga.Total)),
            (SagaStep.ReservingStock, OutOfStock) =>
                Go(SagaStep.Cancelled, null, new CancelOrder(id, "OutOfStock")),
            (SagaStep.ReservingStock, SagaTimedOut) =>
                Go(SagaStep.Cancelled, null, new ReleaseStock(id), new CancelOrder(id, "StockTimeout")),

            // Pivot: do not blindly undo a payment with an unknown outcome, ask first
            (SagaStep.CapturingPayment or SagaStep.CheckingPayment, PaymentCaptured) =>
                Go(SagaStep.CreatingShipment, ShipmentTimeout, new MarkOrderPaid(id), new CreateShipment(id)),
            (SagaStep.CapturingPayment or SagaStep.CheckingPayment, PaymentFailed) =>
                Go(SagaStep.ReleasingStock, StockTimeout, new ReleaseStock(id)),
            (SagaStep.CapturingPayment or SagaStep.CheckingPayment, SagaTimedOut) =>
                Go(SagaStep.CheckingPayment, PaymentTimeout, new CheckPaymentStatus(id)),

            // Retryable: no rollback, retry; when attempts run out, tell a human
            (SagaStep.CreatingShipment, ShipmentCreated) =>
                Go(SagaStep.Completed, null),
            (SagaStep.CreatingShipment, SagaTimedOut) when saga.Attempts < MaxShipmentAttempts =>
                Go(SagaStep.CreatingShipment, ShipmentTimeout, new CreateShipment(id)),
            (SagaStep.CreatingShipment, SagaTimedOut) =>
                Go(SagaStep.Failed, null, new AlertOperations(id, "Shipment could not be created")),

            (SagaStep.ReleasingStock, StockReleased) =>
                Go(SagaStep.Cancelled, null, new CancelOrder(id, "PaymentFailed")),
            (SagaStep.ReleasingStock, SagaTimedOut) =>
                Go(SagaStep.ReleasingStock, StockTimeout, new ReleaseStock(id)),

            // A payment arriving after the saga closed: the money must be refunded
            (SagaStep.Cancelled, PaymentCaptured captured) =>
                Go(SagaStep.Refunding, PaymentTimeout, new RefundPayment(id, captured.Amount)),
            (SagaStep.Refunding, PaymentRefunded) =>
                Go(SagaStep.Cancelled, null),
            (SagaStep.Refunding, RefundFailed failed) =>
                Go(SagaStep.Failed, null, new AlertOperations(id, $"Refund failed: {failed.Reason}")),

            // Everything else (duplicates, unexpected order): the state does not change
            _ => null
        };
    }

    private static Decision Go(SagaStep next, TimeSpan? timeout, params IOrderMessage[] commands) =>
        new(next, timeout, commands);
}

This is the transition table from the "When the Status Column Is Updated from Everywhere" post of the Solution Patterns series, written as code: each line allows one (step, message) pair, everything not on the list is rejected. I chose a single switch over the state objects from the "State Pattern" post of the Design Patterns series, because here behaviour varies by transition rather than by step, and seeing every transition on one screen is the easiest way to review a saga. The final _ => null looks ordinary but does half the work: a second PaymentCaptured reaching a saga in CreatingShipment changes nothing. The saga's idempotency does not come from a separate inbox table but from the question "does this message mean anything in this step?". This code is tested without a database, with nothing but calls to Decide(state, message).

The Handler: Load, Decide, Save in the Same Transaction

The shell around the pure function follows the same four steps for every message:

public sealed class OrderSagaHandler(SagaDbContext db, TimeProvider clock)
{
    public async Task HandleAsync(IOrderMessage message, CancellationToken ct)
    {
        DateTime now = clock.GetUtcNow().UtcDateTime;
        OrderSagaState? saga = await db.Sagas.SingleOrDefaultAsync(s => s.OrderId == message.OrderId, ct);

        if (saga is null)
        {
            if (message is not OrderPlaced placed)
            {
                Trace.Write($"no saga for {message.GetType().Name}, ignored");
                return;
            }
            saga = OrderSagaState.Start(placed, now);
            db.Sagas.Add(saga);
            Enqueue(new ReserveStock(placed.OrderId, placed.Books));
            Trace.Write($"OrderPlaced: started -> {saga.Step}");
            await db.SaveChangesAsync(ct); // a second arrival of the same OrderId violates the primary key
            return;
        }

        SagaStep before = saga.Step;
        Decision? decision = OrderSagaMachine.Decide(saga, message);
        if (decision is null)
        {
            Trace.Write($"{message.GetType().Name} in {saga.Step}: ignored");
            return;
        }

        saga.Apply(decision, now);
        foreach (IOrderMessage command in decision.Commands)
            Enqueue(command);
        Trace.Write($"{message.GetType().Name}: {before} -> {saga.Step}" +
                    (decision.Commands.Count > 0 ? $" => {string.Join(", ", decision.Commands.Select(c => c.GetType().Name))}" : ""));

        // State and commands in the same transaction (the outbox from post 07); throws if Version does not match
        await db.SaveChangesAsync(ct);
    }

    private void Enqueue(IOrderMessage command) => db.Outbox.Add(new OutboxCommand
    {
        Type = command.GetType().Name,
        Payload = JsonSerializer.Serialize(command, command.GetType())
    });
}

Commands do not go straight to the broker; they are written to an outbox table in the same SaveChangesAsync as the saga state. This is the dual write problem from the series' "Transactional Outbox and Inbox" post, and it is even more dangerous in a saga: if the state says "waiting for payment" but CapturePayment is never sent, the saga waits forever, and if it is sent but the state is not written, the saga will not recognize the reply. The handler calls no external system; the HTTP (Hypertext Transfer Protocol) request to the payment provider is made by Payments' own consumer. The saga's transaction lasts a few milliseconds and waits on nobody's network. The happy path and a duplicate arriving afterwards:

A) Happy path, then a duplicate PaymentCaptured
  [saga] OrderPlaced: started -> ReservingStock
  [saga] StockReserved: ReservingStock -> CapturingPayment => CapturePayment
  [saga] PaymentCaptured: CapturingPayment -> CreatingShipment => MarkOrderPaid, CreateShipment
  [ordering] order -> Paid
  [saga] ShipmentCreated: CreatingShipment -> Completed
  [saga] PaymentCaptured in Completed: ignored
  final: Completed, version 3, deadline none

Correlation and Optimistic Concurrency

If messages are delivered at least once and processed by several consumer instances, two messages for the same saga can land on two threads at the same time. The most dangerous pair is the payment reply and the payment timeout: if the reply arrived exactly at second 30, both read the same state. I played this out step by step with two DbContext instances:

    await using SagaDbContext first = createDb();
    await using SagaDbContext second = createDb();
    OrderSagaState a = await first.Sagas.SingleAsync(s => s.OrderId == id);
    OrderSagaState b = await second.Sagas.SingleAsync(s => s.OrderId == id);
    Console.WriteLine($"  both handlers loaded version {a.Version}, step {a.Step}");

    a.Apply(OrderSagaMachine.Decide(a, new PaymentCaptured(id, 60m))!, now);
    await first.SaveChangesAsync();
    Console.WriteLine($"  handler 1 (PaymentCaptured) saved: {a.Step}, version {a.Version}");

    b.Apply(OrderSagaMachine.Decide(b, new SagaTimedOut(id, SagaStep.CapturingPayment))!, now);
    try
    {
        await second.SaveChangesAsync();
    }
    catch (DbUpdateConcurrencyException)
    {
        Console.WriteLine("  handler 2 (SagaTimedOut) lost the race: DbUpdateConcurrencyException");
    }
    await inbox.DeliverAsync(new SagaTimedOut(id, SagaStep.CapturingPayment), ct);

D) PaymentCaptured and the payment timeout race
  [saga] OrderPlaced: started -> ReservingStock
  [saga] StockReserved: ReservingStock -> CapturingPayment => CapturePayment
  [payments] captured, but the reply was lost
  both handlers loaded version 1, step CapturingPayment
  handler 1 (PaymentCaptured) saved: CreatingShipment, version 2
  handler 2 (SagaTimedOut) lost the race: DbUpdateConcurrencyException
  [saga] SagaTimedOut in CreatingShipment: ignored
  final: CreatingShipment, version 2, deadline 10:05:31

Because Version is a concurrency token, EF Core sent the second UPDATE with WHERE Version = 1; the row was already at 2, so zero rows were affected and DbUpdateConcurrencyException was thrown. The general mechanism of concurrency tokens is covered in the EF Core series' "Concurrency in EF Core" post. Without optimistic concurrency the second handler would have written CheckingPayment over CreatingShipment, leaving a saga that queries an already captured payment and forgets the shipment: a classic lost update. The right reaction for the losing side is not to drop the message but to retry with fresh state:

// Entry point that delivers messages to the saga: retries with fresh state on a concurrency conflict
public sealed class SagaInbox(Func<SagaDbContext> createDb, TimeProvider clock)
{
    public async Task DeliverAsync(IOrderMessage message, CancellationToken ct)
    {
        for (int attempt = 1; ; attempt++)
        {
            await using SagaDbContext db = createDb();
            try
            {
                await new OrderSagaHandler(db, clock).HandleAsync(message, ct);
                return;
            }
            catch (DbUpdateConcurrencyException) when (attempt < 3)
            {
                Trace.Write($"{message.GetType().Name}: concurrency conflict, retrying with fresh state");
            }
        }
    }
}

On the retry, the timeout found the saga in CreatingShipment and the first line of Decide kicked in: the timeout message carries the step it was set for (Step), and once the step has changed, the old timeout is meaningless. MassTransit's schedule token does the same job. If conflicts are frequent (many messages per second for one saga), an alternative is to route the messages of the same OrderId to a single partition or session, as in the series' "Message Ordering and Partitioning" post, giving one consumer per saga. In Bookstore an order receives a few messages per minute; optimistic concurrency is enough.

Timeouts: A Deadline Column or Scheduled Messages?

A saga's real enemy is not failure but silence: a step that fails at least produces an event, a step that never answers keeps the saga waiting forever. Every waiting step needs a deadline. There are two common implementations. The first is polling that periodically scans the deadline in the saga state:

// Polling for timeouts: delivers SagaTimedOut to sagas whose deadline has passed
public sealed class SagaTimeoutScanner(Func<SagaDbContext> createDb, SagaInbox inbox, TimeProvider clock)
{
    public async Task<int> FireDueAsync(CancellationToken ct)
    {
        DateTime now = clock.GetUtcNow().UtcDateTime;
        List<SagaTimedOut> due;
        await using (SagaDbContext db = createDb())
        {
            due = await db.Sagas
                .Where(s => s.DeadlineUtc != null && s.DeadlineUtc <= now)
                .Select(s => new SagaTimedOut(s.OrderId, s.Step))
                .ToListAsync(ct);
        }
        foreach (SagaTimedOut timedOut in due)
            await inbox.DeliverAsync(timedOut, ct);
        return due.Count;
    }
}

The second is leaving a delayed message on the broker when entering the step: Azure Service Bus scheduled messages, or a scheduler such as Quartz.NET or Hangfire. MassTransit's Schedule and Wolverine's TimeoutMessage abstract this path. Polling's advantage is that the deadline sits on the same row and in the same transaction as the saga state: when the step changes, the deadline changes too, and there is no separate scheduled message to cancel. Its drawbacks are that resolution depends on the scan interval and that the table needs an index. For an order saga that does not need second-level precision, polling is simple and sufficient; in tests, moving time forward with FakeTimeProvider takes one line.

The real question is what to do on a timeout. Blind compensation at the pivot is the most expensive mistake a saga can make: a saga that says "the payment did not answer within 30 seconds, release the stock and cancel the order" takes the customer's money and cancels their book whenever the payment was actually captured but the reply got lost on the way. The state machine in the example moves to CheckingPayment on a pivot timeout and asks the payment provider for the status:

B) The provider captured, but its reply was lost
  [saga] OrderPlaced: started -> ReservingStock
  [saga] StockReserved: ReservingStock -> CapturingPayment => CapturePayment
  [payments] captured, but the reply was lost
  31 s later the timeout scanner runs
  [saga] SagaTimedOut: CapturingPayment -> CheckingPayment => CheckPaymentStatus
  [saga] PaymentCaptured: CheckingPayment -> CreatingShipment => MarkOrderPaid, CreateShipment
  [ordering] order -> Paid
  [saga] ShipmentCreated: CreatingShipment -> Completed
  final: Completed, version 4, deadline none

For this to work, the payment request must carry an idempotency key: the provider must not capture twice for the same order and must be able to answer a status query by that key. We covered the HTTP side in the "Idempotency Keys" post of the Solution Patterns series; the saga's OrderId is a natural candidate for that key. This is also the saga-side counterpart of the target from the Introduction to Architecture series' quality attribute scenarios, "if the payment provider does not answer within 5 seconds, the order becomes PaymentPending and the user gets no error": the user is put on hold, and the saga resolves the uncertainty.

Compensation Can Fail Too

The "Other Errors" section of Garcia-Molina and Salem's paper asks the question most designs still skip: what if the compensating transaction itself fails? If retrying would hit the same error, the system can neither roll back nor move forward. The worst scenario in the example is exactly that: the payment request seems never to have reached the provider, the saga releases the stock and cancels the order, then the provider processes the in-flight request after all, and the refund is rejected too.

C) Request never arrived; the capture shows up late and the refund fails
  [saga] OrderPlaced: started -> ReservingStock
  [saga] StockReserved: ReservingStock -> CapturingPayment => CapturePayment
  [payments] request never reached the provider
  31 s later the timeout scanner runs
  [saga] SagaTimedOut: CapturingPayment -> CheckingPayment => CheckPaymentStatus
  [saga] PaymentFailed: CheckingPayment -> ReleasingStock => ReleaseStock
  [inventory] stock released
  [saga] StockReleased: ReleasingStock -> Cancelled => CancelOrder
  [ordering] order -> Cancelled (PaymentFailed)
  ...the provider processes the original request after all
  [saga] PaymentCaptured: Cancelled -> Refunding => RefundPayment
  [saga] RefundFailed: Refunding -> Failed => AlertOperations
  [ops] ALERT: Refund failed: card account closed
  final: Failed, version 6, deadline none

Two design decisions are visible here. First, the saga keeps listening after it moves to Cancelled; deleting a closed saga from the table right away means losing the late PaymentCaptured as "no saga, ignore" and silently keeping the customer's money. So completed saga rows must be kept for as long as late messages can arrive (the payment provider's longest processing time, the DLQ (dead-letter queue) replay window). Second, Failed is a terminal state, and it is for a human: the AlertOperations command raises an alert, and the support team contacts the customer and returns the money some other way. A retryable step running out of attempts ends up in the same place:

E) The carrier API stays down
  [saga] OrderPlaced: started -> ReservingStock
  [saga] StockReserved: ReservingStock -> CapturingPayment => CapturePayment
  [saga] PaymentCaptured: CapturingPayment -> CreatingShipment => MarkOrderPaid, CreateShipment
  [ordering] order -> Paid
  [shipping] carrier API timed out
  [saga] SagaTimedOut: CreatingShipment -> CreatingShipment => CreateShipment
  [shipping] carrier API timed out
  [saga] SagaTimedOut: CreatingShipment -> CreatingShipment => CreateShipment
  [shipping] carrier API timed out
  [saga] SagaTimedOut: CreatingShipment -> Failed => AlertOperations
  [ops] ALERT: Shipment could not be created
  final: Failed, version 5, deadline none

Since the pivot has passed, the saga does not roll back; after three attempts it hands over to a human. In a real system the time between attempts grows exponentially, and "three" is chosen from the carrier's typical outage length. The number of sagas in Failed is one of the architectural signals we will discuss in the last post of the series.

The Same Saga in MassTransit and Wolverine

Libraries hand you this skeleton ready-made. MassTransit's state machine writes the same saga in a DSL (domain-specific language):

// Persisted state: CorrelationId and CurrentState are MassTransit's contract
public sealed class OrderSagaState : SagaStateMachineInstance, ISagaVersion
{
    public Guid CorrelationId { get; set; }
    public string CurrentState { get; set; } = "";
    public decimal Total { get; set; }
    public Guid? PaymentTimeoutTokenId { get; set; }
    public int Version { get; set; }
}

        During(ReservingStock,
            When(Reserved)
                .Publish(ctx => new CapturePayment(ctx.Saga.CorrelationId, ctx.Saga.Total))
                .Schedule(PaymentTimeout, ctx => new PaymentTimeoutExpired(ctx.Saga.CorrelationId))
                .TransitionTo(CapturingPayment));

        During(CapturingPayment, CheckingPayment,
            When(Captured)
                .Unschedule(PaymentTimeout)
                .Publish(ctx => new CreateShipment(ctx.Saga.CorrelationId))
                .TransitionTo(CreatingShipment),
            When(Declined)
                .Unschedule(PaymentTimeout)
                .Publish(ctx => new ReleaseStock(ctx.Saga.CorrelationId))
                .TransitionTo(ReleasingStock));

        // Timeout at the pivot: no blind compensation, ask for the payment status first
        During(CapturingPayment,
            When(PaymentTimeout.Received)
                .Publish(ctx => new CheckPaymentStatus(ctx.Saga.CorrelationId))
                .TransitionTo(CheckingPayment));

after OrderPlaced: in ReservingStock = True, ReserveStock published: True
after StockReserved: in CapturingPayment = True
after PaymentFailed: in ReleasingStock = True, ReleaseStock published: True
after StockReleased: finalized = True

The output is from MassTransit's in-memory test harness. ISagaVersion covers optimistic concurrency, Schedule/Unschedule the timeout token, and During(...) the step matching in our switch; an undefined (step, event) pair is treated as an error by default and ignored explicitly with Ignore(...). License note: this code was built with 8.5.12 (Apache 2.0); MassTransit 9.x is commercially licensed, and according to Massient's page v8 support ends on 31 December 2026. Moving from 8 to 9 is a topic for the Backend series.

In Wolverine (MIT license) the saga is one class, holding state properties and handler methods together:

// Wolverine saga: state and handlers in the same class; Wolverine manages persistence
public sealed class OrderSaga : Saga
{
    public Guid Id { get; set; }
    public SagaStep Step { get; set; }
    public decimal Total { get; set; }

    public static (OrderSaga, ReserveStock) Start(OrderPlaced placed) =>
        (new OrderSaga { Id = placed.OrderId, Step = SagaStep.ReservingStock, Total = placed.Total },
         new ReserveStock(placed.OrderId, placed.Books));

    public (CapturePayment, PaymentTimeout) Handle([SagaIdentityFrom(nameof(StockReserved.OrderId))] StockReserved reserved)
    {
        Step = SagaStep.CapturingPayment;
        return (new CapturePayment(Id, Total), new PaymentTimeout(Id));
    }

    public CheckPaymentStatus? Handle([SagaIdentityFrom(nameof(PaymentTimeout.OrderId))] PaymentTimeout timeout)
    {
        if (Step != SagaStep.CapturingPayment)
            return null; // the payment reply already arrived
        Step = SagaStep.CheckingPayment;
        return new CheckPaymentStatus(Id);
    }

Happy path: 60.00 EUR
  [inventory] reserved 2 x Dune
  [payments] capture 60.00 EUR
  [shipping] shipment created
  handled: ReserveStock -> StockReserved -> CapturePayment -> PaymentCaptured -> CreateShipment -> ShipmentCreated
Payment declined: 135.00 EUR
  [inventory] reserved 2 x Dune
  [payments] capture 135.00 EUR
  [inventory] released
  handled: ReserveStock -> StockReserved -> CapturePayment -> PaymentFailed -> ReleaseStock -> StockReleased

In Wolverine the values a handler returns (tuples included) are sent as cascading messages; PaymentTimeout derives from TimeoutMessage(30.Seconds()), so it is delivered with a delay. Our messages name the saga id OrderId, so [SagaIdentityFrom] tells Wolverine which property to use; its default convention looks for OrderSagaId, SagaId or Id. MarkCompleted() deletes the saga row; remember the late message discussion above, because when to call it is a design decision. The example ran on Wolverine's in-memory saga persistence; in production you would use EF Core, Marten, or Wolverine's JSON (JavaScript Object Notation) saga table on SQL Server/PostgreSQL. In the 6.62.0 version I used, the core package no longer ships the runtime compiler; you need to add the WolverineFx.RuntimeCompilation package or pre-generate the code, which I learned from the error message on the first run.

Which one? Writing it from scratch is a few hundred lines and you know every one of them; a library comes with an outbox, retries, scheduler integration and a test harness. In a system with one or two sagas, code from scratch makes sense; in one with dozens, a library does.

When Is a Workflow Engine Worth It?

As a saga's steps multiply, waits that last hours or days (when will the customer send the return parcel?), human approvals, and the need to change the flow's version while instances are running start to appear. At that point a workflow engine is considered instead of a hand-written state machine. Temporal's .NET SDK (Software Development Kit; MIT license) and Microsoft's Durable Task family (Azure Durable Functions and the open-source Durable Task .NET SDK, MIT license) let you write a workflow as ordinary async C# code; the engine records the result of every await in an event history and, after a crash, "replays" the code from the start to get back to where it was. The cost comes from that model: workflow code must be deterministic (no direct DateTime.UtcNow, random numbers or I/O), changing the code while workflows are running requires versioning, and the engine itself is new infrastructure to operate. For Bookstore's three-step order saga this is too much; if a flow such as returns, lasting days and involving human steps, comes along, the decision should be revisited.

Common Pitfalls

Mixing saga state into the domain model

Adding PaymentTimeoutTokenId, CurrentSagaStep and ShipmentAttempts to the Order entity fills the domain model with the details of a workflow engine. The domain knows Order.Status (PaymentPending, Paid); the saga knows which message it is waiting for. They answer different questions.

Calling the external system inside the handler

If the saga handler calls the payment provider over HTTP directly, it waits on the network with a transaction open, and the call's outcome and the state write are once again two separate operations. The saga only decides and writes commands; the participant makes the external call.

Not tying a timeout to a step

A timeout that carries no step information, arriving after the saga has moved on, is mistaken for the new step's timeout. The example's SagaTimedOut(OrderId, Step), or MassTransit's token, prevents that.

Deleting a completed saga right away

For late messages, the saga's final state must stay known for a while longer. Keeping the final state and archiving it periodically is safer than deleting it.

A saga for everything

Turning work that could finish in one database and one transaction into a saga adds complexity with nothing in return. The "When Not to Use It" section of the previous post applies here too.

A Small Experiment

★☆☆ Write a few tests for OrderSagaMachine.Decide without a database: PaymentCaptured in the CreatingShipment step must return null, and SagaTimedOut in CapturingPayment must produce CheckPaymentStatus. How many lines of code does it take?

★★☆ Add the customer's cancellation request to the saga: if a CancelRequested message arrives in ReservingStock, release the stock and cancel the order; in CapturingPayment, record the request and wait for the payment outcome; in CreatingShipment, cancel with a refund. How did the user experience change compared with the previous post's semantic lock?

★★★ Run two SagaInbox instances with Task.WhenAll, delivering hundreds of messages to the same saga at the same time (on PostgreSQL rather than SQLite, for instance). How many concurrency conflicts occur, and are three attempts enough? How does the number change when the messages of the same OrderId are routed to a single worker?

The saga now survives crashes, silence and concurrent messages. But the contents of the messages will change over time: when a currency field is added to PaymentCaptured, or a field of OrderPlaced is renamed, what happens to consumers running the old version? In the next post we look at message contracts, schema evolution, CloudEvents and AsyncAPI.