Trade-off Thinking: No Right Answer, Only the Least Worst Option

Version note: The code in this post was built and run with C# 14 and .NET 10.

Guess First

When an order is placed in Bookstore, the customer gets a confirmation email. Today's code is in its most natural form: the order is saved, then the email is sent within the same request, and the order number is returned. We simulate the email provider with a fake and try four cases: the SMTP (Simple Mail Transfer Protocol) server is normal (150 ms), slow (2 seconds), down, and down while the customer presses "try again".

using System.Diagnostics;

OrderStore store = new();
FakeEmailSender email = new();
OrderService orders = new(store, email);

email.Delay = TimeSpan.FromMilliseconds(150);
await Measure("smtp normal", () => orders.PlaceOrderAsync("Ada", 44.47m));

email.Delay = TimeSpan.FromSeconds(2);
await Measure("smtp slow", () => orders.PlaceOrderAsync("Alan", 12.99m));

email.Delay = TimeSpan.FromMilliseconds(150);
email.IsDown = true;
await Measure("smtp down", () => orders.PlaceOrderAsync("Grace", 18.49m));
await Measure("retry", () => orders.PlaceOrderAsync("Grace", 18.49m));

Console.WriteLine($"orders in database: {store.Count} (Grace: {store.CountFor("Grace")})");

static async Task Measure(string label, Func<Task<int>> placeOrder)
{
    Stopwatch watch = Stopwatch.StartNew();
    try
    {
        int id = await placeOrder();
        Console.WriteLine($"{label,-12} order {id} placed in {watch.ElapsedMilliseconds} ms");
    }
    catch (Exception ex)
    {
        Console.WriteLine($"{label,-12} FAILED after {watch.ElapsedMilliseconds} ms: {ex.Message}");
    }
}

public sealed record OrderConfirmation(int OrderId, string CustomerName);

public interface IEmailSender
{
    Task SendAsync(OrderConfirmation message, CancellationToken ct = default);
}

public sealed class OrderService(OrderStore store, IEmailSender email)
{
    public async Task<int> PlaceOrderAsync(string customerName, decimal total, CancellationToken ct = default)
    {
        int id = store.Save(customerName, total);
        // Order is saved; the email goes synchronously in the same request
        await email.SendAsync(new OrderConfirmation(id, customerName), ct);
        return id;
    }
}

public sealed class OrderStore
{
    private readonly List<(int Id, string CustomerName, decimal Total)> _rows = [];

    public int Count => _rows.Count;

    public int Save(string customerName, decimal total)
    {
        int id = 1042 + _rows.Count;
        _rows.Add((id, customerName, total));
        return id;
    }

    public int CountFor(string customerName) => _rows.Count(r => r.CustomerName == customerName);
}

public sealed class FakeEmailSender : IEmailSender
{
    public TimeSpan Delay { get; set; }
    public bool IsDown { get; set; }

    public async Task SendAsync(OrderConfirmation message, CancellationToken ct = default)
    {
        await Task.Delay(Delay, ct);
        if (IsDown)
            throw new IOException("SMTP server unavailable");
    }
}

How many milliseconds does each line take? After Grace's two attempts, what does the customer see and how many orders are in the database?

The Real Cost of Synchronous Email

smtp normal  order 1042 placed in 155 ms
smtp slow    order 1043 placed in 2002 ms
smtp down    FAILED after 152 ms: SMTP server unavailable
retry        FAILED after 152 ms: SMTP server unavailable
orders in database: 4 (Grace: 2)

There are three separate problems, and none of them is about the email itself. The first is latency: checkout now takes as long as the email provider. When the provider slowed down, Alan's order took two seconds; the "checkout p95 < 800 ms" scenario from the previous post is at the mercy of a system we don't control. The second is availability: when the provider is down, the customer sees an error. Checkout's availability is now roughly the database's availability times the email provider's; if both are 99.9 percent, the product is about 99.8 percent, close to 86 minutes of downtime a month instead of 43. The third, and the sneakiest, is consistency: the exception was thrown after the order was saved. The customer saw "failed", tried again, and the database ended up with two orders from Grace. This is exactly the duplicate order problem from the "Idempotency Key" post in the Solution Patterns series; this time it comes not from a double click but from the architecture.

This kind of dependency is called temporal coupling: two parts have to be up at the same time. Placing an order is far more important than sending an email, but because both are tied to the same request, the unimportant one can bring down the important one.

Handing It to a Queue

The classic fix is to hand the email job to a queue, return from the request without waiting, and let a background worker send the email. In .NET the simplest in-process queue is System.Threading.Channels. This time we start the scenario with SMTP down, bring the server back after a while, and finally stop the process while emails are being sent:

using System.Diagnostics;
using System.Threading.Channels;

OrderStore store = new();
FakeEmailSender email = new() { Delay = TimeSpan.FromMilliseconds(150), IsDown = true };
Channel<OrderConfirmation> queue = Channel.CreateBounded<OrderConfirmation>(1_000);
OrderService orders = new(store, queue.Writer);
EmailWorker worker = new(queue.Reader, email);

using CancellationTokenSource shutdown = new();
Task running = worker.RunAsync(shutdown.Token);

await Measure("smtp down", () => orders.PlaceOrderAsync("Grace", 18.49m));
await Measure("smtp down", () => orders.PlaceOrderAsync("Linus", 9.99m));
await Task.Delay(1_000);
Console.WriteLine($"after 1 s: sent={email.SentCount}, attempts={worker.Attempts}, queued={queue.Reader.Count}");

email.IsDown = false;
await Task.Delay(1_000);
Console.WriteLine($"smtp back:  sent={email.SentCount}, queued={queue.Reader.Count}");

email.Delay = TimeSpan.FromSeconds(2);
for (int i = 0; i < 3; i++)
    await orders.PlaceOrderAsync("Barbara", 7.50m);
await Task.Delay(100);

// Deployment or crash: the process stops, taking the in-memory queue with it
shutdown.Cancel();
await running;
Console.WriteLine($"process stopped: orders={store.Count}, sent={email.SentCount}, " +
    $"lost={queue.Reader.Count + worker.InFlight}");

static async Task Measure(string label, Func<Task<int>> placeOrder)
{
    Stopwatch watch = Stopwatch.StartNew();
    int id = await placeOrder();
    Console.WriteLine($"{label,-10} order {id} placed in {watch.ElapsedMilliseconds} ms");
}

public sealed record OrderConfirmation(int OrderId, string CustomerName);

public interface IEmailSender
{
    Task SendAsync(OrderConfirmation message, CancellationToken ct = default);
}

public sealed class OrderService(OrderStore store, ChannelWriter<OrderConfirmation> emails)
{
    public async Task<int> PlaceOrderAsync(string customerName, decimal total, CancellationToken ct = default)
    {
        int id = store.Save(customerName, total);
        // The email job is handed to the queue; the request doesn't wait
        await emails.WriteAsync(new OrderConfirmation(id, customerName), ct);
        return id;
    }
}

public sealed class EmailWorker(ChannelReader<OrderConfirmation> queue, IEmailSender email)
{
    public int Attempts { get; private set; }
    public int InFlight { get; private set; }

    public async Task RunAsync(CancellationToken ct)
    {
        try
        {
            await foreach (OrderConfirmation message in queue.ReadAllAsync(ct))
            {
                InFlight = 1;
                while (true)
                {
                    try
                    {
                        Attempts++;
                        await email.SendAsync(message, ct);
                        break;
                    }
                    catch (IOException)
                    {
                        // If SMTP is down, wait and retry the same message
                        await Task.Delay(300, ct);
                    }
                }
                InFlight = 0;
            }
        }
        catch (OperationCanceledException)
        {
            // Shutdown: the message in hand and everything in the queue go away with the memory
        }
    }
}

public sealed class OrderStore
{
    private readonly List<(int Id, string CustomerName, decimal Total)> _rows = [];

    public int Count => _rows.Count;

    public int Save(string customerName, decimal total)
    {
        int id = 1042 + _rows.Count;
        _rows.Add((id, customerName, total));
        return id;
    }
}

public sealed class FakeEmailSender : IEmailSender
{
    private int _sent;

    public TimeSpan Delay { get; set; }
    public bool IsDown { get; set; }
    public int SentCount => _sent;

    public async Task SendAsync(OrderConfirmation message, CancellationToken ct = default)
    {
        await Task.Delay(Delay, ct);
        if (IsDown)
            throw new IOException("SMTP server unavailable");
        Interlocked.Increment(ref _sent);
    }
}

smtp down  order 1042 placed in 1 ms
smtp down  order 1043 placed in 0 ms
after 1 s: sent=0, attempts=3, queued=1
smtp back:  sent=2, queued=0
process stopped: orders=5, sent=2, lost=3

Checkout now takes a millisecond; orders are accepted even while SMTP is down. The worker retries the failed send every 300 milliseconds, and when the server comes back the two pending emails go out. There are no errors on the customer's side and no duplicate orders.

The last line shows the price. Barbara's three orders are in the database, but their emails were in the in-memory queue. When the process stopped (a deployment, a crash, a container restart), three emails silently disappeared. No log, no error. The system is also more complex now: a worker, a retry policy, the need to monitor how full the queue is, and a new intermediate state, "order placed but its email not sent".

Two Options, One Table

Let's put the two options side by side. The measurements come from the two programs above:

There is no "correct" column in the table. The decision for Bookstore: the order confirmation email matters but its loss can be compensated (the customer can see the order page, support can resend it), while checkout availability and latency are among the top three characteristics. The queue wins, and the risk of loss is accepted deliberately. The same decision could go the other way for work where loss is unacceptable, such as capturing a payment.

There is also a way to eliminate the loss: write the email request to a table in the same database transaction as the order and have the worker read from there. It is called the transactional outbox; it solves the loss but brings its own trade-offs, such as writing to and cleaning up the database and the chance of duplicate emails on resend. The details are in the upcoming Messaging and Event-Driven Architecture series.

Why Do Characteristics Conflict?

In this example, availability and performance conflicted with consistency (the email really going out together with the order). Such conflicts are the rule, not the exception. A few familiar pairs:

Security and performance. Storing passwords with a slow hash raises an attacker's brute-force cost but also slows down every login; the iteration count in the password storage post of the OWASP (Open Worldwide Application Security Project) Top 10 series is exactly the knob for this trade-off. Authorization checks, encryption and audit logging performed on every request add to latency too.

Flexibility and simplicity. Making every payment provider a loadable plugin makes adding a new provider easier, but makes the code harder to read today, when there is only one provider. The "speculative generality" discussion in the SOLID (Single Responsibility, Open/Closed, Liskov Substitution, Interface Segregation, Dependency Inversion) and Design Principles series is the class-level counterpart of this.

Consistency and availability. The "PaymentPending" decision in the previous post was a small example: rejecting the order when the payment provider doesn't respond is consistent but loses the sale; accepting it saves the sale but creates, for a while, a state called "order with uncertain payment" in the system. In distributed systems the formal name of this tension is the CAP (Consistency, Availability, Partition tolerance) theorem; the details are in the upcoming System Design and Distributed Systems Fundamentals series.

.NET's own library had to make the same decision. The console logger doesn't write log messages directly; it puts them in an in-memory queue and a background thread writes them. What happens when the queue is full is a setting:

using Microsoft.Extensions.Logging.Console;

ConsoleLoggerOptions defaults = new();
Console.WriteLine($"MaxQueueLength={defaults.MaxQueueLength}, QueueFullMode={defaults.QueueFullMode}");

MaxQueueLength=2500, QueueFullMode=Wait

The default is Wait: if the queue is full, the thread writing the log waits, so logs are not lost but the application slows down. Choose DropWrite and the application never waits, but excess messages are dropped. The .NET team chose to sacrifice latency rather than lose logs by default and left the choice to you as a setting. That is what a good trade-off decision looks like: the cost of both options is known, the default has a reason, and changing it is possible.

Reversible and Irreversible Decisions

Treating every trade-off decision with the same seriousness is a mistake too. In Amazon's shareholder letter for 2015, Jeff Bezos splits decisions in two. Type 1 decisions are one-way doors: once through, going back is impossible or very expensive; they should be made slowly, carefully and with consultation. Type 2 decisions are two-way doors: if it turns out wrong, you walk back; they should be made quickly by small groups or individuals. The letter's warning is that as organizations grow, they start making Type 2 decisions with the heavy Type 1 process too, which brings slowness and fear of experimenting.

Let's look at Bookstore's email decision this way. Using an in-process Channel<T> is a two-way door: OrderService only sees a ChannelWriter<OrderConfirmation>, and replacing it tomorrow with an implementation that writes to a durable queue concerns a single class. In contrast, choosing a message broker product and turning the message format into a contract other teams consume is much closer to a one-way door; once that format spreads, like the order number in the first post, it has crossed the system boundary.

The practical consequence: walk through two-way doors quickly, slow down in front of one-way doors. Better still, look for ways to turn a one-way door into a two-way one: an interface, an adapter, a feature flag, separating the contract exposed to the outside from the model inside.

The Last Responsible Moment and YAGNI

When should a one-way decision be made? One of the tools in Mary and Tom Poppendieck's Lean Software Development (2003) is the last responsible moment: delaying a decision until the moment when not deciding would eliminate an important alternative. The value of delaying is that you gather more information until then. Bookstore takes 800 orders a day today; we don't know which broker will be needed, the message volume, or which teams will consume these events. Choosing a broker today means making the most expensive decision with the least information.

But the word "responsible" in "last responsible" matters. If you delay beyond that moment, the decision gets made for you: the system grows, synchronous email sending is copied into ten places, and going back becomes expensive. This is where the balance with YAGNI (You Aren't Gonna Need It) is struck; as we saw in the DRY (Don't Repeat Yourself), KISS (Keep It Simple, Stupid) and YAGNI post of the SOLID and Design Principles series, YAGNI says don't build a feature you don't need today, not give up on keeping code changeable. Bookstore's choice is an example of that balance: an in-process queue today (cheap, sufficient), but a boundary where OrderService doesn't know how the queue is implemented (a small investment that keeps a future move to a broker a two-way door).

As a rule: if keeping an option open is cheap (an interface, a method signature), keep it; if it is expensive (building and operating distributed infrastructure from day one), wait, and write down when you will look again.

Writing Down the "Why" of a Decision

In the first post we saw the second law of FoSA (Fundamentals of Software Architecture, Richards and Ford): why is more important than how. This decision will be questioned six months from now; someone will ask "why do emails sometimes disappear?". The answer needs to be in a written note, not in the code. In its smallest form a decision note contains:

• Context: checkout latency and availability are among the top three characteristics; the email provider slows down once or twice a week.
• Decision: the order confirmation email will be sent in the background through an in-process queue.
• Alternatives: synchronous sending; a transactional outbox; an external message broker.
• Accepted cost: queued emails may be lost during a deployment or a crash; the customer can see the status on the order page.
• Revisit when: losses exceed 10 emails a month, or another team wants to consume order events.

The last item is the most often skipped and the most valuable: it says in advance under which condition the decision stops being valid. The standard form of such a note is the ADR (Architecture Decision Record); its format, where it is kept and how to keep it alive in a team are the subject of the upcoming Architecture Decisions series.

Common Pitfalls

Not seeing the trade-off. "Let's put it on a queue, it'll be faster" has no cost in it. A proposal without its cost is half a proposal; that is exactly the warning of FoSA's first law.

Treating every decision as a one-way door. A three-week architecture board process to try an in-process queue slows the team down and kills the culture of experimenting. Make reversible decisions quickly.

Walking through a one-way door without noticing. Exposing a message format, a public URL (Uniform Resource Locator) or a database schema to other teams often happens silently. Before exposing something beyond the system boundary, stop and ask what going back would cost.

Mistaking YAGNI for giving up on changeability. Not setting up a broker today may be right; wiring OrderService directly to an SMTP client closes the door for later. Keep the cheap options open.

Not measuring the probability of loss. Don't assume the in-memory queue "rarely" loses messages; log the number of messages left in the queue at shutdown. If you don't know what the cost you accepted actually is, you haven't really accepted it.

A Small Experiment

★☆☆ In the queued program, set the capacity of Channel.CreateBounded to 2 and place five orders in a row while SMTP is down. How long does PlaceOrderAsync take? Which option does a full bounded queue push checkout back to?

★★☆ Add shutdown behaviour to the worker: when shutdown is cancelled, write the messages left in the queue and the one in hand to a file, and when the process starts again read them back into the queue. How much does this reduce the loss, and in which case is there still loss?

★★★ Find a decision made in your own system in the last year and discuss whether it is a two-way or a one-way door. If it is one-way, what would be the smallest change to turn it into a two-way door? Rewrite the decision with the five-item note above.

This post closes Block A: we covered what architecture is, characteristics, measuring them and choosing between them. In the next post we move to the first architecture style: Layered Architecture, the closest to Bookstore's current shape, and the real responsibilities of the presentation, business and data layers.