Running YARP in Production: Observability, Performance, Containers and Multiple Instances

Version note: The code in this post was built and run with .NET 10, Yarp.ReverseProxy 2.3 and OpenTelemetry .NET 1.21; the load test used k6 2.4 and traces were collected with Jaeger 2.23.

Guess First

You run Bookstore.Gateway as two instances behind a load balancer that hands out requests in turn. The limit from post 04 still applies: Ada can send at most 5 order requests per 10 seconds. If Ada sends 12 requests within 10 seconds, how many get a 429?

Second question: while a request that takes 8 seconds is in flight at the gateway, Kubernetes, or you, sends SIGTERM to the gateway. What does the client waiting for that request see: a 200 or a connection error? Assume you haven't touched any of the gateway's settings.

How a Trace Passes Through the Gateway: OpenTelemetry

Once the gateway is in production, the first question is "where is the slowness?". Bookstore, the online bookstore used as the shared example across this site's series, now consists of a gateway, a BFF and five services behind them; finding which step of a request lost time from log lines is hard. What trace context is, the traceparent header and adding the TraceId to logs were covered in the Logging series' post on correlation IDs and trace context. Here we look at the gateway-specific part: YARP's own spans and the trace passing through the proxy unbroken.

public static IServiceCollection AddGatewayTelemetry(this IServiceCollection services, IConfiguration configuration)
{
    OpenTelemetryBuilder otel = services.AddOpenTelemetry()
        .ConfigureResource(resource => resource.AddService("bookstore-gateway"))
        .WithTracing(tracing => tracing
            .AddAspNetCoreInstrumentation()
            .AddHttpClientInstrumentation()
            // YARP's own spans: proxy.forwarder per proxied request, plus active health check probes
            .AddSource("Yarp.ReverseProxy"))
        .WithMetrics(metrics => metrics
            .AddAspNetCoreInstrumentation()
            .AddHttpClientInstrumentation());

    // export only when a collector is configured (OTEL_EXPORTER_OTLP_ENDPOINT)
    if (!string.IsNullOrEmpty(configuration["OTEL_EXPORTER_OTLP_ENDPOINT"]))
    {
        otel.UseOtlpExporter();
    }
    return services;
}

I added a few lines of the same setup, with the same packages, to Catalog and Inventory. I sent everything to Jaeger over OTLP and pulled two requests' traces out of Jaeger's API as trees:

docker run -d --name yarpseries-jaeger -p 4327:4317 -p 16696:16686 jaegertracing/jaeger:2.23.0
# start every service with OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4327

bookstore-gateway    GET /api/catalog/{**rest}             35.3 ms
  bookstore-gateway    proxy.forwarder                       15.4 ms
    bookstore-gateway    GET                                    3.0 ms
      bookstore-catalog    GET /books/{isbn}                      2.4 ms

bookstore-gateway    GET /api/bff/books/{isbn}             69.0 ms
  bookstore-gateway    GET                                    2.7 ms
    bookstore-catalog    GET /books/{isbn}                      2.0 ms
  bookstore-gateway    GET                                   37.6 ms
    bookstore-inventory  GET /stock/{isbn}                     24.6 ms
  bookstore-gateway    GET                                   44.4 ms

The first trace is a proxy route. Under ASP.NET Core's server span sits YARP's proxy.forwarder span, under it the HTTP request the forwarder sends to the backend, and at the bottom Catalog's own server span. All four spans carry the same TraceId; the YARP forwarder adds traceparent to the backend request itself, and we didn't have to write anything. The proxy.forwarder span only appears when AddSource("Yarp.ReverseProxy") is added; YARP doesn't create the activity at all if nobody listens. The second trace is the aggregation endpoint from post 05: you can see at a glance that the three calls run in parallel and that the total is close to the slowest call (Ordering, 44 ms). The last GET has no backend span beneath it because I didn't add OpenTelemetry to Ordering. The trace ends there, but traceparent is still sent; the day you instrument Ordering, the tree completes itself.

The large gaps in the durations (a 35 ms server span against a 2.4 ms backend) are the warm-up cost of the first requests. We measure what the gateway adds under steady traffic with a load test below. One warning: YARP also produces spans for active health check probes. With our settings that's three probes every 5 seconds, tens of thousands of short traces a day. You need sampling, or filtering of those span names (proxy.cluster_health_checks, proxy.destination_health_check) in the collector. Metrics, sampling and collector setup are covered in detail in the upcoming Observability series.

YARP's Counters and ForwarderError

A trace tells the story of one request; counters tell you the gateway's overall state. YARP 2.3 publishes request counters through an EventSource named Yarp.ReverseProxy. I read them with dotnet-counters during the load test:

dotnet-counters collect -p <gateway-pid> --counters Yarp.ReverseProxy --format csv --duration 00:00:00:05

Timestamp,Provider,Counter Name,Counter Type,Mean/Increment
11/14/2026 14:35:26,Yarp.ReverseProxy,Requests Started,Metric,1073140
11/14/2026 14:35:26,Yarp.ReverseProxy,Requests Started Rate (Count / 1 sec),Rate,2001
11/14/2026 14:35:26,Yarp.ReverseProxy,Requests Failed,Metric,0
11/14/2026 14:35:26,Yarp.ReverseProxy,Current Requests,Metric,0

These are EventCounters; they don't flow into OpenTelemetry metrics directly. In production, the ASP.NET Core and System.Net.Http metrics (the WithMetrics above) are usually enough, because every incoming and outgoing request of the gateway already passes through them. If you need finer per-request timing (connecting, sending headers, copying the body), the Yarp.Telemetry.Consumption package offers interfaces for listening to these events in your own code.

Behind the "Requests Failed" counter is ForwarderError. We've been writing this value into our log line since post 01. Based on the definitions in YARP's source code, these are the ones you'll see most:

Alert on the error kind, not the status code. If 502s go up, whether they're Request or ResponseBodyDestination goes to different teams: the first is network or deployment, the second is the backend's own bug. The same log line also works as an access log: route, cluster, destination, status and error kind. If you use ASP.NET Core's HTTP logging middleware, don't turn on request and response body logging; every token, piece of personal data and file at the gateway lands in the log, and log volume starts competing with the traffic itself.

How Much Latency Does the Gateway Add?

We promised this measurement in post 01. I called the same Catalog endpoint with k6, first directly and then through the gateway. For the gateway route I picked catalog-legacy, which has no cache and no timeout; all apps were Release builds with log level Warning, all of them and k6 on the same machine.

import http from 'k6/http';
import { check } from 'k6';

// usage: k6 run -e URL=... -e RATE=2000 rate.js  (fixed arrival rate, not a saturation test)
export const options = {
  scenarios: {
    fixed: {
      executor: 'constant-arrival-rate',
      rate: Number(__ENV.RATE),
      timeUnit: '1s',
      duration: '20s',
      preAllocatedVUs: 100,
    },
  },
};

export default function () {
  const res = http.get(__ENV.URL);
  check(res, { 'status is 200': (r) => r.status === 200 });
}

# 2000 requests per second, 20 seconds
== http://localhost:5301/books/9780441172719
    http_req_duration..............: med=78µs    p(95)=133µs   p(99)=521µs
    http_reqs......................: 40000  1999.937002/s
== http://localhost:5300/api/catalog/books/9780441172719
    http_req_duration..............: med=151µs    p(95)=191µs    p(99)=1.07ms
    http_reqs......................: 40001  1999.979201/s

# saturation test: 50 virtual users, 20 seconds, as fast as possible
== http://localhost:5301/books/9780441172719
    http_req_duration..............: med=317µs    p(95)=722µs   p(99)=1.43ms
    http_reqs......................: 2485218 124257.979937/s
== http://localhost:5300/api/catalog/books/9780441172719
    http_req_duration..............: med=871µs    p(95)=1.66ms p(99)=2.68ms
    http_reqs......................: 1016953 50838.39487/s

Under normal load the gateway adds about 70 µs at the median and about 0.5 ms at p99. Next to the total time of a web page, that difference is too small to notice. The saturation test changes the picture: on the same machine the gateway path carried 51 thousand requests per second, the direct path 124 thousand. These numbers don't mean "the gateway is three times slower". Every request now goes through two HTTP exchanges, and five processes (k6, the gateway and three Catalogs) share the same CPU; the gateway's cost is in CPU more than in latency. When you measure in your own environment, run the gateway on a separate machine and start with a fixed-rate test; load test design is covered in detail in the upcoming Performance and Load Testing series.

Most of the settings that decide performance are on the backend side. According to YARP's documentation, each cluster has its own HttpMessageInvoker, and connections are reused from that invoker's pool; a new connection isn't opened per request. If the backend speaks HTTP/2, many requests can share one connection, but as we saw in post 03, unencrypted HTTP/2 needs explicit settings on both sides. On body size, Kestrel's default limit applies at the gateway too; YARP's MaxRequestBodySize route setting changes it per route (according to the source code comment, -1 removes the limit). Kestrel's limits and settings against slow clients were covered in the ASP.NET Core Security series' post on Kestrel and request limits.

Containers: Our Own Image and YARP's Ready-Made Image

The shortest way to put the gateway in a container is the .NET SDK's own container support; it produces an image without a Dockerfile. As in post 05, the Angular files are built into wwwroot first:

cd web/bookstore-web && npm install && npx ng build && cd ../..
dotnet publish src/Bookstore.Gateway -c Release -t:PublishContainer \
  -p:ContainerRepository=bookstore-gateway -p:ContainerImageTag=1.0.0
  Building image 'bookstore-gateway' with tags '1.0.0' on top of base image 'mcr.microsoft.com/dotnet/aspnet:10.0'.
  Pushed image 'bookstore-gateway:1.0.0' to local registry via 'docker'.

docker run -d --name yarpseries-gateway -p 5400:8080 \
  -e Auth__SigningKey=dev-only-bookstore-signing-key-not-a-secret-0123456789 \
  -e DestinationAddresses__catalog-1=http://host.docker.internal:5301/ \
  -e DestinationAddresses__catalog-2=http://host.docker.internal:5311/ \
  -e DestinationAddresses__catalog-3=http://host.docker.internal:5312/ \
  -e Services__ordering__http__0=host.docker.internal:5302 \
  bookstore-gateway:1.0.0

The image is based on aspnet:10.0, listens on 8080 and runs as the non-root app user (uid 1654). The signing key here is a fake example; in reality it comes from a secret store. Addresses are passed as environment variables through the DestinationAddressFilter from post 02, and on the first run the container didn't see these settings:

curl http://localhost:5400/api/v1/books/9780441172719
  502
warn: Yarp.ReverseProxy.Health.ActiveHealthCheckMonitor[17]
      Probing destination `catalog-1[localhost:5301]` on cluster `catalog` failed.

The environment variables were in the container; the problem was ordering. According to YARP's source code, the service discovery destination resolver we added in post 03 runs before config filters and renames the destination ID from catalog-1 to catalog-1[localhost:5301]. The filter then looks up the key DestinationAddresses:catalog-1[localhost:5301] and doesn't find it. The filter from post 02 had silently stopped working the day the resolver was added in post 03; locally the addresses were already right, so nobody noticed. The fix is to look up by the part of the ID before the square bracket:

foreach ((string id, DestinationConfig destination) in cluster.Destinations)
{
    // the service discovery resolver runs before config filters and renames "catalog-1"
    // to "catalog-1[localhost:5301]"; look the override up by the configured id
    string configuredId = id.Split('[')[0];
    string? address = configuration[$"DestinationAddresses:{configuredId}"];
    destinations[id] = string.IsNullOrEmpty(address)
        ? destination
        : destination with { Address = address };
}

{"isbn":"9780441172719","title":"Dune","author":"Frank Herbert","price":420.00}  200

The destination ID in the log still reads catalog-2[localhost:5311], but requests go to host.docker.internal; keep that in mind when reading logs. The lesson: try every setting that changes per environment at least once for real, inside the container, in an environment that resembles production.

The YARP team also has a ready-made container image: mcr.microsoft.com/dotnet/nightly/yarp. It runs a gateway from a JSON file, with no code:

docker run -d --name yarpseries-yarp -p 5401:5000 \
  -v "$(pwd)/deploy/yarp-image/yarp.json:/etc/yarp.config:ro" \
  mcr.microsoft.com/dotnet/nightly/yarp:2.3.0-preview.5

HTTP/1.1 200 OK X-Catalog-Instance: catalog-1
HTTP/1.1 200 OK X-Catalog-Instance: catalog-2
HTTP/1.1 200 OK X-Catalog-Instance: catalog-1

Inspecting the image makes the decision clear. The repository lives only under nightly, and its tags are preview and latest; there's no stable version tag. Inside are Yarp.ReverseProxy 2.3.0, service discovery, OpenTelemetry with an OTLP exporter, and static file support switched on by an environment variable (YARP_ENABLE_STATIC_FILES). It runs on a small base without a shell, as a non-root user; it reads its config from /etc/yarp.config and listens on 5000. But it runs on the .NET 9.0.23 runtime, and .NET 9 support ended on 10 November 2026. There's also no extension point for authentication, rate limiting, output caching or our own transforms. This is the image that Aspire's AddYarp resource uses; it's practical for local development and simple routing, but a gateway that contains code, like the one in this series, needs its own image. The upcoming Distributed Apps with Aspire series covers the Aspire side.

Multiple Instances and Graceful Shutdown

In production the gateway shouldn't be a single instance; a single gateway becomes a single point of failure for the whole system. I started two instances (5300 and 5320) and tried the first question from "Guess First":

# Ada, 12 requests, alternating between two instances
5300:200 5320:200 5300:200 5320:200 5300:200 5320:200 5300:200 5320:200 5300:200 5320:200 5300:429 5320:429

# output cache, two requests per instance
X-Catalog-Instance: catalog-1  <- :5300
Age: 0 X-Catalog-Instance: catalog-1  <- :5300
X-Catalog-Instance: catalog-1  <- :5320
Age: 0 X-Catalog-Instance: catalog-1  <- :5320

The answer is 2: Ada got 10 requests through instead of 5. What we said in posts 02 and 04 is now measured: rate limiter and output cache counters live in the instance's memory. With output caching, each instance goes to the backend separately; that's usually acceptable. With rate limits, though, the limit is effectively multiplied by the number of instances. Solutions such as a shared store or dividing the limit by the instance count were discussed in the ASP.NET Core Security series' post on rate limiting. Distributing the route config across instances is the same question: every instance that reads from a file sees its own file, while the database provider from post 02 has every instance read the same table every 10 seconds. The container's log also has a warning that DataProtection keys are stored inside the container. It doesn't matter today, but if the gateway starts issuing cookies (like the OpenID Connect cookie session mentioned in post 05), the keys must be shared across instances.

The second question is graceful shutdown. I started Catalog with an 8 second delay, sent a request to the gateway, and one second later sent SIGTERM to the gateway process:

# default settings
SIGTERM at +1s
new request at +2s: curl exit 7
long request: 200 after 8.065999s
      Application is shutting down...

# ASPNETCORE_SHUTDOWNTIMEOUTSECONDS=3
SIGTERM at +1s
new request at +2s: curl exit 7
long request: 000 after 4.017660s
long request: curl exit 56

With the defaults, the open request got a 200 at the end of its 8 seconds. On SIGTERM the host stopped accepting new connections (the new request got a connection error) and waited for the open requests. The wait is ShutdownTimeout; according to the docs its default has been 30 seconds since .NET 6. When I lowered it to 3 seconds, the time ran out and the open request ended with a dropped connection. In Kubernetes this should line up with the pod's termination period (terminationGracePeriodSeconds). Removing the pod from the endpoint list also starts at the same time as SIGTERM; for a few seconds new requests can still reach the pod that's shutting down, which is why a short wait before starting to shut down is a common fix. For long-lived connections such as WebSockets, 30 seconds is often not enough; those connections need to be able to reconnect from the client side.

Whether to put the gateway behind a Kubernetes Ingress or Gateway API, or in place of them, is a separate decision. Putting it behind them leaves TLS and external traffic to the platform's standard component, and the gateway only does work that needs code; the cost is one more proxy hop and routing in two places. Putting it in their place gives you a single routing layer, but you have to manage TLS certificates and external load balancing yourself. Ingress and Gateway API were covered in the Kubernetes series' post on external traffic. If you put it behind them, watch the forwarded header chain: YARP writes X-Forwarded-For with the client address the gateway sees (post 02); if the gateway is behind an ingress, that address is the ingress's. The gateway's ForwardedHeaders middleware has to be set up to trust only the ingress's network; the details are in the Web Servers and Hosting series' post on forwarded headers.

Production Notes

Define an SLO for the gateway itself and measure it separately from the backends. Errors the gateway produces itself (NoAvailableDestinations, Request, rate limit 429s, timeout 504s) and errors passed through from backends belong on separate charts; otherwise every backend problem becomes the gateway team's alert.

Plan capacity by CPU. As we measured, the latency cost is small, but every request at the gateway accepts one HTTP request and opens another. Watch the CPU of the gateway instances and tie autoscaling to it.

Account for health check traffic. Three destinations, a 5 second interval and ten gateway instances mean two probes per second per backend; the health endpoint must be cheap and filtered out of traces.

Think about the order of config and code during deployments. If a new route uses a new policy name, and that policy is defined in code that isn't deployed yet, config coming from the database fails on the old instances. Release the code first, then the config.

Common Pitfalls

First, running the gateway as a single instance. When the gateway goes down, everything goes down; at least two instances and a load balancer with health checks are a must.

Second, logging bodies at the gateway. Personal data, tokens and files land in the log, and cost and risk grow together.

Third, testing per-instance features (rate limits, output caching, random choices without a canary bucket) on a single instance and expecting the same behavior in production.

Fourth, not aligning the shutdown time with the platform. A short ShutdownTimeout cuts open requests during deployments; a very long one ends the same way when Kubernetes kills the pod by force.

Fifth, putting a ready-made image into production without looking at its tag. An image tagged nightly and preview, running on a runtime that's out of support, trades the gain of not writing code for the cost of not getting security updates.

A Small Experiment

★☆☆ Connect the gateway to Jaeger with OTEL_EXPORTER_OTLP_ENDPOINT and add OpenTelemetry to Ordering as in Catalog. How many spans does the trace of the aggregation endpoint from post 05 have now? Which one takes the longest?

★★☆ Run the k6 test against the output-cached /api/v1/books route and compare it with the direct Catalog result. Why might the gateway be faster this time? Repeat the test with an Authorization header and explain the difference using what you learned in post 04.

★★★ Add a readiness endpoint to the gateway bound to IHostApplicationLifetime.ApplicationStopping: when SIGTERM arrives, the endpoint returns 503 immediately, and the host starts shutting down only after waiting 5 seconds. Put a second YARP with active health checks, as in post 03, in front of two gateway instances, and measure during a load test that no new requests reach the instance that's shutting down.

The gateway can now be observed, has a measured cost and runs as a container. In the last post of the series we look at YARP's history: the defaults that changed from 1.0 to 2.3, reading an old YARP configuration, and how concepts map when moving an Ocelot-based gateway to YARP.