Cross-Cutting Concerns at the Gateway: Authentication, Authorization, Rate Limiting, Timeouts and Output Caching

Version note: The code in this post was built and run with .NET 10, Yarp.ReverseProxy 2.3 and Microsoft.AspNetCore.Authentication.JwtBearer 10.0.

Guess First

In Bookstore.Gateway you attached a 2 second request timeout policy to the book read route and set the status code returned on timeout to 504. One of the Catalog instances answers in 3 seconds. Which status code does a client that lands on that instance see after 2 seconds? If you said 504, the real answer with this YARP and .NET 10 pair is different.

Second question: what happens if you move app.UseOutputCache() above app.UseAuthorization()? No compile error, the app runs. Think about which route could hand whose response to whom.

Attaching Policies to Routes, and Middleware Order

What makes a gateway valuable for Bookstore, the online bookstore used as the shared example across this site's series, is gathering work that every service would otherwise repeat, such as identity, traffic limits and caching, in one place. YARP doesn't invent a new mechanism for this: it uses ASP.NET Core's own middleware, and in route config you only give the policy name. When YARP adds the route as an endpoint, it turns those names into endpoint metadata, and the middleware treats the proxy route exactly as it would treat a normal controller endpoint.

"catalog-v1": {
  "ClusterId": "catalog",
  "AuthorizationPolicy": "Anonymous",
  "CorsPolicy": "bookstore-web",
  "TimeoutPolicy": "catalog-read",
  "OutputCachePolicy": "catalog-list",
  "Match": {
    "Path": "/api/v1/books/{**rest}"
  },
  "Transforms": [
    { "PathPattern": "/books/{**rest}" }
  ]
},
"orders-v1": {
  "ClusterId": "ordering",
  "AuthorizationPolicy": "customer",
  "CorsPolicy": "bookstore-web",
  "RateLimiterPolicy": "per-user-orders",
  "Match": {
    "Path": "/api/v1/orders/{**rest}"
  },
  "Transforms": [
    { "PathRemovePrefix": "/api/v1" }
  ]
}

Some of these route properties came later: according to the release notes, RateLimiterPolicy was added in 2.0, Timeout and TimeoutPolicy in 2.1, and OutputCachePolicy in 2.2. On an older YARP version, writing these fields in config does nothing. What matters on our side is the middleware order:

var app = builder.Build();

// order matters: CORS answers preflights, authentication sets User for the rate limiter
// partition, authorization rejects before anything reaches the cache or the backend
app.UseCors();
app.UseAuthentication();
app.UseRateLimiter();
app.UseAuthorization();
app.UseRequestTimeouts();
app.UseOutputCache();

app.MapGet("/gateway/ping", () => "pong").AllowAnonymous();

app.MapReverseProxy(proxyPipeline => { /* the pipeline from post 01 */ });

I put the rate limiter after authentication but before authorization. It has to come after authentication because we partition by user, so it needs to know the user. It comes before authorization so that unauthenticated requests are counted too; otherwise an attacker getting 401s could keep trying without ever hitting a limit. Output cache comes after authorization; that's the answer to the second "Guess First" question. If output cache came before authorization, a cached response could be returned without authorization ever running. It makes no difference on an anonymous route like our book list, but the day someone adds a cache policy to a route that requires identity, this order hands that response to unauthenticated clients too. We covered the logic of middleware order in the Middleware Pipeline post of the ASP.NET Core Web API series; here the same rule becomes traffic security.

JWT Validation at the Edge

The gateway validates JWTs like any ASP.NET Core app. We covered issuing and validating tokens in the JWT post of the ASP.NET Core Web API series, and validating access tokens from a real identity provider in the Authentication and Authorization series post on protecting an API. Here we use a fake symmetric key for development; it lives only in appsettings.Development.json, and as its name says, it's not a secret. The token comes from tools/Bookstore.DevToken in the repo; the commands below use bash syntax, and in PowerShell $ADA = dotnet run --project tools/Bookstore.DevToken -- ada is enough.

public static IServiceCollection AddGatewayAuthentication(
    this IServiceCollection services, IConfiguration configuration)
{
    IConfigurationSection auth = configuration.GetSection("Auth");

    services.AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
        .AddJwtBearer(options =>
        {
            options.MapInboundClaims = false;
            options.TokenValidationParameters = new TokenValidationParameters
            {
                ValidIssuer = auth["Issuer"],
                ValidAudience = auth["Audience"],
                IssuerSigningKey = new SymmetricSecurityKey(Encoding.UTF8.GetBytes(auth["SigningKey"]!)),
                NameClaimType = "name",
                RoleClaimType = "role",
            };
        });

    services.AddAuthorizationBuilder()
        // a route without AuthorizationPolicy must not become public by accident
        .SetFallbackPolicy(new AuthorizationPolicyBuilder().RequireAuthenticatedUser().Build())
        .AddPolicy(CustomerPolicy, policy => policy.RequireAuthenticatedUser().RequireRole("Customer"));

    return services;
}

A route's AuthorizationPolicy is either a policy name or one of YARP's two special values: Default (ASP.NET Core's default policy, an authenticated user) and Anonymous. For anything else, that is, routes with no AuthorizationPolicy, the fallback policy kicks in. Its default is null, so a route you didn't annotate is public. We made the fallback "authenticated user" and wrote Anonymous explicitly on the Catalog routes that must be public. In post 02 we wrote a provider that reads routes from a database; the catalog-v2 row in that table has no policy at all. Thanks to the fallback:

anon v2: 401
token v2: 200

A new route the operations team adds to the table doesn't end up on the internet because nobody thought about it. Now let's look at orders with four kinds of users:

ADA=$(dotnet run --project tools/Bookstore.DevToken -- ada)
STAFF=$(dotnet run --project tools/Bookstore.DevToken -- grace StoreStaff)

curl -i http://localhost:5300/api/v1/orders
HTTP/1.1 401 Unauthorized
Content-Length: 0
WWW-Authenticate: Bearer

curl -H "Authorization: Bearer $ADA" http://localhost:5300/api/v1/orders
[{"id":1001,"customerId":"ada","customerEmail":"ada@example.com","total":605.50,"status":"Shipped"}]

curl -o /dev/null -w "%{http_code}\n" -H "Authorization: Bearer $ADA" http://localhost:5300/api/v1/orders/1002
404

curl -i -H "Authorization: Bearer $STAFF" http://localhost:5300/api/v1/orders
HTTP/1.1 403 Forbidden

The unauthenticated request got 401 at the gateway and never reached the backend. grace is authenticated but not in the Customer role, hence 403. Ada sees her own order; when she looks at Alan's order 1002 she gets 404. That 404 comes from Ordering, not from the gateway, because "does this order belong to this user" is a business rule and not the gateway's job. When I sent a token with a broken signature, the response was:

HTTP/1.1 401 Unauthorized
WWW-Authenticate: Bearer error="invalid_token", error_description="The signature key was not found"

Carrying Identity to the Backend: Token Passthrough and Internal Headers

The gateway authenticated the user. How does Ordering know who that user is? There are two ways, and we discussed their trade-offs in the Microservices series post on security in microservices. Here we look at how they look on the YARP side.

The first is token passthrough. Since YARP copies headers by default, the Authorization header reaches the backend on its own. Ordering validates the same token itself:

// the gateway already validated the token, Ordering validates it again
IConfigurationSection auth = builder.Configuration.GetSection("Auth");
builder.Services.AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
    .AddJwtBearer(options =>
    {
        options.MapInboundClaims = false;
        options.TokenValidationParameters = new TokenValidationParameters
        {
            ValidIssuer = auth["Issuer"],
            ValidAudience = auth["Audience"],
            IssuerSigningKey = new SymmetricSecurityKey(Encoding.UTF8.GetBytes(auth["SigningKey"]!)),
            NameClaimType = "name",
            RoleClaimType = "role",
        };
    });

RouteGroupBuilder group = app.MapGroup("/orders").RequireAuthorization();

group.MapGet("/", (ClaimsPrincipal user) =>
    orders.Where(o => o.CustomerId == user.FindFirstValue("sub")));

curl -o /dev/null -w "%{http_code}\n" http://localhost:5302/orders
401

When I go to Ordering directly, bypassing the gateway, I also get 401. This is defense in depth: if the network is misconfigured, an internal service calls Ordering directly, or someone exposes the port, Ordering still doesn't accept an unauthenticated request. The cost is that every service needs to know the token validation settings; with a real identity provider, those settings are just the issuer address.

The second is an internal header. The gateway reads the user ID from the validated token and passes it to the backend in a header it sets itself. In post 02 we wrote a transform that strips X-Bookstore- prefixed headers on the way in; now that transform adds the user ID:

// only after authentication: the value comes from a validated token, never from the client
string? userId = transformContext.HttpContext.User.FindFirstValue("sub");
if (userId is not null)
{
    transformContext.ProxyRequest.Headers.TryAddWithoutValidation(InternalHeaders.UserId, userId);
}

I sent a request with Ada's token while trying to look like Alan:

curl "http://localhost:5300/debug/catalog" -H "Authorization: Bearer $ADA" -H "X-Bookstore-User-Id: alan"
{
    "path": "/debug/request",
    "query": "",
    "host": "localhost:5300",
    "headers": {
        "X-Bookstore-Route": "catalog-debug",
        "X-Bookstore-User-Id": "ada",
        "X-Forwarded-For": "::1",
        "X-Forwarded-Host": "localhost:5300",
        "X-Forwarded-Proto": "http"
    }
}

The alan the client wrote was removed and replaced with ada from the token. The internal header approach simplifies the backend, but it's only safe under one condition: the backend can't be reached any way other than through the gateway, or the backend verifies that the request came from the gateway, for example with mutual TLS. In Bookstore, Ordering validates the token itself, and we use the internal header as extra information for logs and auditing.

Centralizing Rate Limiting at the Gateway

We put a per-user limit on the orders route: 5 requests per 10 seconds. Unauthenticated requests are grouped by client IP and get a stricter limit: 2 per 10 seconds.

services.AddRateLimiter(options =>
{
    options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;
    options.AddPolicy(PerUserOrders, context =>
    {
        // authenticated users get their own bucket; everyone else is grouped by client IP
        string? userId = context.User.FindFirstValue("sub");
        return userId is not null
            ? RateLimitPartition.GetFixedWindowLimiter($"user:{userId}", _ => new FixedWindowRateLimiterOptions
            {
                PermitLimit = 5,
                Window = TimeSpan.FromSeconds(10),
                QueueLimit = 0,
            })
            : RateLimitPartition.GetFixedWindowLimiter($"ip:{context.Connection.RemoteIpAddress}", _ => new FixedWindowRateLimiterOptions
            {
                PermitLimit = 2,
                Window = TimeSpan.FromSeconds(10),
                QueueLimit = 0,
            });
    });
    options.OnRejected = (context, _) =>
    {
        if (context.Lease.TryGetMetadata(MetadataName.RetryAfter, out TimeSpan retryAfter))
        {
            context.HttpContext.Response.Headers.RetryAfter =
                ((int)Math.Ceiling(retryAfter.TotalSeconds)).ToString(CultureInfo.InvariantCulture);
        }
        return ValueTask.CompletedTask;
    };
});

# Ada, 7 requests in a row
200
200
200
200
200
Retry-After: 10
429
Retry-After: 10
429

# Alan, at the same time
200

# no token, 4 requests
401 401 429 429

Ada got 429 after the fifth request, and the Retry-After header tells her how long to wait; Alan, at the same time, went on unaffected in his own bucket. Requests without a token got 401 first, then 429: because the rate limiter runs before authorization, unauthenticated attempts are counted too. Choosing an algorithm (fixed window, sliding window, token bucket, concurrency), designing partition keys and chained limiters are covered in detail in the Rate Limiting post of the ASP.NET Core Security series. From the gateway's point of view, the most important thing to know is this: these counters live in the gateway instance's memory. With three gateway instances, Ada can effectively send 15 requests per 10 seconds. If you need strict consistency, you need approaches like a shared store (Redis) or dividing the limit by the instance count; that post covers this too.

Be careful with IP when choosing a partition key. If the gateway sits behind a load balancer, RemoteIpAddress is the load balancer's address and every unauthenticated user lands in one bucket. You first need to configure the ForwardedHeaders middleware with a list of trusted proxies; we covered that in the forwarded headers post of the Web Servers and Hosting series.

Timeouts: Per-Route Limits and an Unexpected 400

In post 03 we saw the cluster's ActivityTimeout: the longest time that may pass with no data from the backend. For a total time limit per route we use the request timeouts middleware that came with .NET 8:

services.AddRequestTimeouts(options =>
{
    options.AddPolicy(CatalogRead, new RequestTimeoutPolicy
    {
        Timeout = TimeSpan.FromSeconds(2),
        TimeoutStatusCode = StatusCodes.Status504GatewayTimeout,
    });
});

A route can take a policy name with TimeoutPolicy or a duration directly with Timeout; setting both gets the config rejected. I started catalog-2 with a 3 second delay and sent book requests:

200 0.001913s
200 0.000970s
400 2.014835s
400 2.003367s

The answer to the first "Guess First" question: 400 Bad Request. The timeout fired at the right moment, at 2 seconds, but the status code is wrong. The reason is in YARP's source code. When the time runs out, the request timeouts middleware cancels the HttpContext.RequestAborted token. When YARP's forwarder sees that cancellation, it interprets it as "the client disconnected", reports the error as RequestCanceled, and writes 400. Since no exception is thrown, the timeout middleware's 504 is never applied. On top of that, client cancellations have been logged at Debug level since 2.2, so you see nothing in the log either. On the client side, 400 means "your request is broken"; that's the wrong message for a client that doesn't retry and shows the error to the user.

The fix is a small step in the YARP pipeline that corrects the status after the forwarder:

public static IReverseProxyApplicationBuilder UseRequestTimeoutStatus(this IReverseProxyApplicationBuilder proxyPipeline)
{
    proxyPipeline.Use(async (context, next) =>
    {
        await next();

        bool timedOut = context.Features.Get<IHttpRequestTimeoutFeature>()?.RequestTimeoutToken.IsCancellationRequested == true;
        if (timedOut
            && context.GetForwarderErrorFeature()?.Error == ForwarderError.RequestCanceled
            && !context.Response.HasStarted)
        {
            context.Response.StatusCode = StatusCodes.Status504GatewayTimeout;
        }
    });
    return proxyPipeline;
}

I added this step right behind the logging middleware from post 01, before UseSessionAffinity(), and put the forwarder error kind in the log line:

504 2.009918s
404 0.002555s
504 2.002800s

      Forwarded to catalog-2[localhost:5311], status 504, error RequestCanceled
      Forwarded to catalog-1[localhost:5301], status 404, error none
      Forwarded to catalog-2[localhost:5311], status 504, error RequestCanceled

IHttpRequestTimeoutFeature.RequestTimeoutToken makes the distinction: if the client really left, this token isn't canceled and the 400 stays. Two more notes. Passive health checks don't count client cancellations as destination failures, so a route timeout doesn't take a slow instance out; that's left to the active check. And request timeouts aren't applied while a debugger is attached, whereas ActivityTimeout always applies. Most "the timeout doesn't work" or "the connection drops while debugging" complaints come from this difference.

Output Caching and CORS

The book list is the same for everyone and read often, so we attached a 30 second output cache policy at the gateway:

services.AddOutputCache(options =>
{
    options.AddPolicy(CatalogList, policy => policy.Expire(TimeSpan.FromSeconds(30)).Tag("catalog"));
});

# no token, 3 requests
X-Catalog-Instance: catalog-2
--
Age: 0
X-Catalog-Instance: catalog-2
--
Age: 0
X-Catalog-Instance: catalog-2

# with Ada's token, 2 requests
X-Catalog-Instance: catalog-2
--
X-Catalog-Instance: catalog-1

Of the three requests without a token, only the first went to Catalog (a single Proxying to line in the log); the other two came from the gateway's memory with an Age header. Requests with a token went to the backend every time. That's not a bug: the base policy of ASP.NET Core output caching neither serves requests with an Authorization header from the cache nor stores them. That default is what keeps a personal response from reaching someone else. If you write your own policy and remove that check, say "let's serve the book list from cache to signed-in users too", then on every route where you don't split the cache key per user, Ada's orders can reach Alan. Vary-by, tag-based eviction and the Redis store are in the Output Cache post of the Caching series; my rule for the gateway is simple: only cache responses that are the same for everyone.

For CORS we give the route a CorsPolicy. The policy allows the Angular development server's address:

curl -i -X OPTIONS http://localhost:5300/api/v1/orders -H "Origin: http://localhost:4200" \
  -H "Access-Control-Request-Method: GET" -H "Access-Control-Request-Headers: authorization"
HTTP/1.1 204 No Content
Access-Control-Allow-Headers: Authorization,Content-Type
Access-Control-Allow-Methods: GET,POST
Access-Control-Allow-Origin: http://localhost:4200

curl -i -X OPTIONS http://localhost:5300/api/v1/orders -H "Origin: http://evil.example" \
  -H "Access-Control-Request-Method: GET"
HTTP/1.1 204 No Content

The gateway answered the preflight; the request neither hit authorization nor reached Ordering. The disallowed origin also got 204, but without any Access-Control-Allow-* headers, so the browser won't send the real request. In post 05, when we serve Angular from the same origin as the gateway, we won't need CORS at all; that's one of the advantages of a BFF.

Proxying WebSockets, SignalR and gRPC

YARP proxies WebSocket connections by default; no extra route setting is needed. Over HTTP/1.1 the connection is upgraded with 101 Switching Protocols, and YARP then carries a bidirectional stream. Since 2.0, WebSockets over HTTP/2 are supported too, and the incoming and outgoing HTTP versions can differ. I didn't actually run WebSockets or gRPC in this post; the following comes from the YARP documentation and from our HTTP/2 experiment in post 03.

Timeouts behave differently on long-lived connections. Request timeouts are disabled after the WebSocket handshake, but the cluster's ActivityTimeout applies for as long as the connection stays open: if no data or keep-alive frame passes in either direction, YARP closes the connection. The trap the documentation highlights: browsers don't send keep-alives on their own, the default of ASP.NET Core's WebSocketOptions.KeepAliveInterval is 2 minutes, and YARP's default ActivityTimeout is 100 seconds. So if you don't touch any setting, a quiet WebSocket connection drops after 100 seconds. Either shorten the backend's keep-alive interval or raise that cluster's ActivityTimeout. SignalR sends its own keep-alive every 15 seconds by default, so you usually don't see this with SignalR; its long polling transport, however, brings back the session affinity requirement from post 03.

gRPC relies entirely on HTTP/2, and request timeouts do apply to gRPC requests. As we saw in post 03, HTTP/2 over an unencrypted connection needs explicit settings on both sides: on the gateway's incoming side, the Kestrel endpoint must listen with Http2 or over HTTPS; on the outgoing side, the cluster needs "Version": "2" and a suitable VersionPolicy. For streaming gRPC calls, set the route timeout and the ActivityTimeout to the real duration of the stream too, or a long server streaming call gets cut off in the middle.

Production Notes

Give every policy at the gateway an owner and a measurement. Track 401, 403 and 429 rates per route; a sudden rise in 401s points to an expired signing key or a misconfigured issuer, and a sudden rise in 429s to a wrong partition key (the load balancer IP, for example).

Don't keep the signing key in a file when validating JWTs at the gateway. With a real identity provider, keys are fetched automatically from the Authority address (OpenID Connect discovery) and rotated; the symmetric key in this post is for development only.

Every middleware you add to the gateway adds to the latency of every request. Authentication and the rate limiter are usually in the microsecond range; the real costs are the connections the gateway holds while waiting for responses and the memory used by the output cache. Output caching lives in instance memory by default; in a gateway with multiple instances each instance keeps its own copy, and tag-based eviction only clears that instance.

Common Pitfalls

First, putting business rules in the gateway. "Institutional customers must order at least 20 copies" or "does this order belong to this user" aren't things the gateway should know. The gateway's policies should stay at the level of identity, roles and traffic. If every new business rule starts requiring a gateway deployment, you're heading toward the "god gateway" we described in the API Gateway post of the Microservices series.

Second, caching personal responses. A custom policy that disables the output cache's base policy shows one user's data to another unless the cache key is split per user.

Third, trusting an internal header while leaving the backend directly reachable. X-Bookstore-User-Id only means something when it's guaranteed that the request really came from the gateway.

Fourth, setting a timeout and never testing the actual status code. As you saw above, even with correct config the client can get 400 instead of 504.

Fifth, running without a fallback policy. Forgetting to write AuthorizationPolicy on a new route, especially when routes come from a database or another team, silently makes that route public.

A Small Experiment

★☆☆ Move app.UseRateLimiter() after app.UseAuthorization() and send four requests to the orders route without a token. How did the status codes change? Write in your own words which order is better against which attack.

★★☆ For the orders route, write an orders.read policy that checks a scope instead of RequireRole("Customer") (update the DevToken tool to add a scope claim to the token). Verify that a token without the scope gets 403 and a token with it gets 200.

★★★ You want to cache the book list for signed-in users too, but safely. Write an output cache policy that also allows caching requests with an Authorization header; attach it only to the /api/v1/books route, and show in code that Catalog reads no identity information on that endpoint, to prove the response doesn't vary by user. Then attach the same policy to the orders route by mistake, test whether Ada and Alan see each other's responses, and write down the result.

The gateway now authenticates, limits traffic, cuts off slow requests and caches shared responses. In the next post we put our own endpoints inside the same app: a BFF for the Angular client, an aggregation endpoint that calls Catalog, Inventory and Ordering in parallel for the book detail screen, and partial failure handling.