Rate Limiting in Depth: Algorithms, Partition Keys, Chained Limiters and Multiple Instances

Version note: The code in this post was built and run with .NET 10 (ASP.NET Core 10.0.13 runtime) and the System.Threading.RateLimiting library that ships with the framework.

Guess First

The Bookstore API (Application Programming Interface) limits anonymous requests to 5 per 10 seconds per IP (Internet Protocol) address. The app runs as two instances behind a reverse proxy. A client sends 12 requests within 10 seconds, and the proxy spreads them across the two instances in turn. How many get a 200? Second question: in the same setup, two completely different users send 3 requests each at the same moment. How many get a 429? Neither answer is "5", and both will show up as real output at the end of the post.

In the Web Security series, which covers the OWASP (Open Worldwide Application Security Project) Top 10, the post "OWASP A06: Insecure Design" used rate limiting to stop brute-forcing of coupon codes, and the ASP.NET Core Web API series post on what's new in .NET 7 shows the basic use of the middleware. This post builds on that: the real difference between the algorithms, choosing a partition key, chaining several limiters, a proper 429 response and where the single-instance assumption breaks. How to design a distributed rate limiter is the subject of the "Designing Rate Limiting" post in the System Design series.

Four Algorithms, the Same Traffic

System.Threading.RateLimiting offers four ready-made limiters. To see the difference, let us send the same traffic to three of them: a limit of 4 requests per 2-second window, 4 requests 150 ms before the window ends, then 4 more 300 ms later.

TimeSpan window = TimeSpan.FromSeconds(2);

Dictionary<string, Func<RateLimiter>> limiters = new()
{
    ["fixed window"] = () => new FixedWindowRateLimiter(new FixedWindowRateLimiterOptions
    {
        PermitLimit = 4, Window = window, QueueLimit = 0
    }),
    ["sliding window"] = () => new SlidingWindowRateLimiter(new SlidingWindowRateLimiterOptions
    {
        PermitLimit = 4, Window = window, SegmentsPerWindow = 4, QueueLimit = 0
    }),
    ["token bucket"] = () => new TokenBucketRateLimiter(new TokenBucketRateLimiterOptions
    {
        TokenLimit = 4, TokensPerPeriod = 2, ReplenishmentPeriod = TimeSpan.FromSeconds(1), QueueLimit = 0
    }),
};

foreach ((string name, Func<RateLimiter> create) in limiters)
{
    using RateLimiter limiter = create();
    Stopwatch clock = Stopwatch.StartNew();
    await Task.Delay(TimeSpan.FromMilliseconds(1850));
    string before = Burst(limiter, 4);
    await Task.Delay(TimeSpan.FromMilliseconds(300));
    string after = Burst(limiter, 4);
    Console.WriteLine($"{name,-15} t=1.85s {before}   t=2.15s {after}   accepted={(before + after).Count(c => c == '+')}/8");
}

fixed window    t=1.85s ++++   t=2.15s ++++   accepted=8/8
sliding window  t=1.85s ++++   t=2.15s ----   accepted=4/8
token bucket    t=1.85s ++++   t=2.15s ++--   accepted=6/8

The fixed window accepted twice the traffic at the window boundary: 8 requests within 300 ms. The window counter resets at second 2 and does not remember what happened before. The sliding window splits the window into 4 segments (500 ms each) and counts the total over the last 2 seconds; the requests from second 1.85 are still inside the window, so none of the second group got through. The token bucket allows a burst while the bucket is full and then refills at 2 tokens per second; the 2 tokens added at second 2 let two requests through.

The fourth one, ConcurrencyLimiter, is independent of time: at most N requests at once. Which one to use when:

Queue: Wait or Reject?

Every limiter has QueueLimit and QueueProcessingOrder settings. The second one decides what happens to a request that arrives while the queue is full, and it works differently from what you might expect:

foreach (QueueProcessingOrder order in new[] { QueueProcessingOrder.OldestFirst, QueueProcessingOrder.NewestFirst })
{
    using ConcurrencyLimiter limiter = new(new ConcurrencyLimiterOptions
    {
        PermitLimit = 1, QueueLimit = 2, QueueProcessingOrder = order
    });
    RateLimitLease running = await limiter.AcquireAsync();
    Task<RateLimitLease>[] waiting = [limiter.AcquireAsync().AsTask(), limiter.AcquireAsync().AsTask(), limiter.AcquireAsync().AsTask()];
    await Task.Delay(50);
    string before = string.Join(" ", waiting.Select((t, i) => $"#{i + 1}:{(t.IsCompleted ? (t.Result.IsAcquired ? "ok" : "rejected") : "queued")}"));
    running.Dispose();
    await Task.Delay(50);
    string after = string.Join(" ", waiting.Select((t, i) => $"#{i + 1}:{(t.IsCompleted ? (t.Result.IsAcquired ? "ok" : "rejected") : "queued")}"));
    Console.WriteLine($"{order,-11} while busy: {before}   after release: {after}");
}

OldestFirst while busy: #1:queued #2:queued #3:rejected   after release: #1:ok #2:queued #3:rejected
NewestFirst while busy: #1:rejected #2:queued #3:queued   after release: #1:rejected #2:queued #3:ok

With OldestFirst, the newcomer is rejected when the queue is full. With NewestFirst, the oldest waiter is evicted from the queue and the newcomer is processed first. NewestFirst makes sense in interactive scenarios where you want to favor the newest request the user is still waiting for; rejecting up front is also an option instead of returning 429 later to a client whose oldest request has long since timed out. For rate limiting, a queue is usually a bad idea: every request you hold keeps a connection and memory busy, and under attack it turns the limiter into a self-inflicted DoS (denial of service). Every limiter in this post uses QueueLimit = 0; consider a queue only for lining up expensive work with ConcurrencyLimiter.

Partition Keys: Who Is the Limit For?

A single FixedWindowRateLimiter is everyone's shared counter. PartitionedRateLimiter keeps a separate limiter per partition key, and you derive the key from the request. Bookstore has three kinds of clients: Pro and Free clients with API keys, and anonymous requests. Each API key belongs to a tenant. We build the global limiter in two layers:

options.GlobalLimiter = PartitionedRateLimiter.CreateChained(
    PartitionedRateLimiter.Create<HttpContext, string>(context =>
    {
        ApiClientRegistry registry = context.RequestServices.GetRequiredService<ApiClientRegistry>();
        string tenant = registry.Resolve(context)?.Client.Tenant ?? "anonymous";
        return RateLimitPartition.GetFixedWindowLimiter($"tenant:{tenant}", _ => new FixedWindowRateLimiterOptions
        {
            PermitLimit = 20, Window = TimeSpan.FromSeconds(10), QueueLimit = 0
        });
    }),
    PartitionedRateLimiter.Create<HttpContext, string>(context =>
    {
        ApiClientRegistry registry = context.RequestServices.GetRequiredService<ApiClientRegistry>();
        if (registry.Resolve(context) is (string key, ApiClient client))
        {
            return RateLimitPartition.GetTokenBucketLimiter($"key:{key}", _ => new TokenBucketRateLimiterOptions
            {
                TokenLimit = client.Tier == Tier.Pro ? 10 : 3,
                TokensPerPeriod = client.Tier == Tier.Pro ? 5 : 1,
                ReplenishmentPeriod = TimeSpan.FromSeconds(1),
                QueueLimit = 0
            });
        }

        string ip = context.Connection.RemoteIpAddress?.ToString() ?? "unknown";
        return RateLimitPartition.GetFixedWindowLimiter($"ip:{ip}", _ => new FixedWindowRateLimiterOptions
        {
            PermitLimit = 5, Window = TimeSpan.FromSeconds(10), QueueLimit = 0
        });
    }));

With CreateChained a request passes only if every limiter in the chain grants it; if one rejects, the result is 429. The first layer is there so that all of a tenant's API keys together cannot choke the system; the second layer stops a single key or IP from eating the tenant's whole share. A partition's factory (_ => new ...Options) runs only once per new key; that is where we pick the options based on the client's tier. The prefix embedded in the partition key string (tenant:, key:, ip:) looks like a detail, but it prevents two different kinds of key from accidentally producing the same value.

== anonymous /books x6
200 200 200 200 200 429
== free key x5
200 200 200 429 429

Alan's bucket on the Free tier holds 3 tokens and refills at 1 token per second. The anonymous client gets 5 requests per 10 seconds. Never take the partition key straight from an unvalidated header: if you partition on X-Api-Key without matching it against registered keys, an attacker can put a random key on every request and get a fresh, empty partition each time, piling up thousands of limiters in memory along the way. That is why unknown keys fall into the IP partition.

Endpoint Policies and DisableRateLimiting

The global limiter applies to every request; endpoint-specific rules are defined as policies and run together with the global limiter (global first, then the endpoint's):

options.AddPolicy("login", context =>
    RateLimitPartition.GetSlidingWindowLimiter(
        context.Connection.RemoteIpAddress?.ToString() ?? "unknown",
        _ => new SlidingWindowRateLimiterOptions
        {
            PermitLimit = 10, Window = TimeSpan.FromMinutes(1), SegmentsPerWindow = 6, QueueLimit = 0
        }));

options.AddFixedWindowLimiter("export", o =>
{
    o.PermitLimit = 2;
    o.Window = TimeSpan.FromSeconds(30);
    o.QueueLimit = 0;
});

app.UseRateLimiter();

app.MapGet("/health", () => "ok").DisableRateLimiting();

app.MapGet("/books", () => new[] { "Dune", "Hyperion" });

app.MapGet("/reports/export", () => Results.Ok(new { file = "orders.csv" }))
    .RequireRateLimiting("export");

DisableRateLimiting turns off both the global limiter and the policies; you do not want health checks to get 429s because the load balancer probes frequently. All eight consecutive /health requests returned 200. The export policy, however, hides a trap: AddFixedWindowLimiter("export", ...) creates a single limiter without partitions, so every client shares the same counter:

== export: ada x2 then grace
ada:200 ada:200 grace:429

Ada downloaded two reports and Grace, from another tenant, got a 429 without having sent a single request. Sometimes that is what you want (limiting an expensive job system-wide), but if it was written with "per user" in mind, it is the easiest way to lock out someone else's users. If you want a per-user limit, write a partitioned limiter with AddPolicy.

429, Retry-After and ProblemDetails

With the default settings a rejected request gets 503, not 429. Printing new RateLimiterOptions().RejectionStatusCode gave 503. A 503 means "the server is unwell", and many client libraries and load balancers read it as a health signal. The first job is to set RejectionStatusCode = 429. Then, in OnRejected, we tell the client when it can try again:

private static async ValueTask WriteRejectionAsync(OnRejectedContext context, CancellationToken cancellationToken)
{
    HttpContext http = context.HttpContext;
    int? retryAfter = context.Lease.TryGetMetadata(MetadataName.RetryAfter, out TimeSpan wait)
        ? (int)Math.Ceiling(wait.TotalSeconds)
        : null;
    if (retryAfter is not null)
    {
        http.Response.Headers.RetryAfter = retryAfter.Value.ToString(CultureInfo.InvariantCulture);
    }

    http.RequestServices.GetRequiredService<ILoggerFactory>().CreateLogger("Bookstore.RateLimiting")
        .LogWarning("Rate limit hit on {Path} for {Client}", http.Request.Path,
            http.Request.Headers["X-Api-Key"].FirstOrDefault() ?? http.Connection.RemoteIpAddress?.ToString());

    await http.RequestServices.GetRequiredService<IProblemDetailsService>().WriteAsync(new ProblemDetailsContext
    {
        HttpContext = http,
        ProblemDetails =
        {
            Status = StatusCodes.Status429TooManyRequests,
            Title = "Too many requests",
            Detail = retryAfter is null ? "Slow down." : $"Retry after {retryAfter} seconds."
        }
    });
}

HTTP/1.1 429 Too Many Requests
Retry-After: 10
{"title":"Too many requests","status":429,"detail":"Retry after 10 seconds.","traceId":"00-99d2eb600bd6b3e29b7f53dfdb6d77ca-8059cc99d8de7da0-00"}

Look closely: this request was rejected about one second into the 10-second window, yet Retry-After says 10. In .NET 10, FixedWindowRateLimiter reports the whole window rather than the time to the next window boundary. .NET 11 fixes this so that the RetryAfter metadata reflects the real boundary; we will come back to that change in the last post of the series. For the token bucket the same field gives the time until the next token; the 429 on the Free key came with Retry-After: 1. The metadata is not always available on every limiter (ConcurrencyLimiter cannot know a time), so the TryGetMetadata check is required. With a chained limiter, the rejecting limiter's metadata is carried on the lease.

We log in OnRejected, but writing every rejection at Warning level makes the logging system the second victim during an attack. In production count rejections with metrics and sample the logs. The Microsoft.AspNetCore.RateLimiting meter, available since .NET 8, already does this; we will use it in the series post on observing security events.

Login: A Per-IP Limit Is Not Enough

The login policy allows 10 attempts per minute per IP. An attacker doing credential stuffing, however, does not come from one IP but from thousands, making a few attempts from each. So login needs a second limit keyed on the targeted account. Since the user name is in the request body, we do this inside the endpoint rather than in the middleware, using the limiter directly:

public sealed class LoginThrottle : IDisposable
{
    private readonly PartitionedRateLimiter<string> _limiter = PartitionedRateLimiter.Create<string, string>(userName =>
        RateLimitPartition.GetFixedWindowLimiter(userName, _ => new FixedWindowRateLimiterOptions
        {
            PermitLimit = 5, Window = TimeSpan.FromMinutes(15), QueueLimit = 0
        }));

    public RateLimitLease TryAcquire(string userName) =>
        _limiter.AttemptAcquire(userName.Trim().ToUpperInvariant());

    public void Dispose() => _limiter.Dispose();
}

app.MapPost("/login", (LoginRequest request, LoginThrottle throttle) =>
{
    using RateLimitLease lease = throttle.TryAcquire(request.UserName);
    if (!lease.IsAcquired)
    {
        return Results.Problem(statusCode: StatusCodes.Status429TooManyRequests, title: "Too many login attempts");
    }

    return request.Password == "dev-only-password" ? Results.Ok() : Results.Unauthorized();
}).RequireRateLimiting("login");

Against the local app, I sent wrong passwords for Ada's account from seven different IP addresses (documentation-range addresses simulated with X-Forwarded-For); on the sixth attempt I wrote the user name as " ADA":

== login for ada from 7 different IPs
401 401 401 401 401 429 429

The IP policy never kicked in, because each IP made only one attempt. The per-user-name limit stopped the attacker after the fifth attempt; without the Trim().ToUpperInvariant() normalization, " ADA" would have opened a brand-new partition. This limiter has a price: an attacker can lock Ada's account for 15 minutes. ASP.NET Core Identity's lockout feature makes the same trade-off; measures such as MFA (multi-factor authentication) and CAPTCHA (Completely Automated Public Turing test to tell Computers and Humans Apart), covered in the Web Security series post "OWASP A07: Authentication Failures (1)", complement this limiter rather than replace it.

Production Notes: Multiple Instances and Proxies

Now the two opening questions. First: two instances, 5 requests per 10 seconds per IP, requests distributed in turn:

200 200 200 200 200 200 200 200 200 200 429 429

The limit became 10, not 5. These limiters live in process memory; each instance keeps its own counter. N instances multiply the limit by N, and if autoscaling changes N, the limit itself changes. Your options: use an approximate value by dividing the limit by the instance count (traffic spreads unevenly without sticky sessions), use a limiter that keeps a shared counter in a store such as Redis (a network call per request), or move the coarse limit to an API gateway or Nginx and keep only business-rule limits (login, coupons) in the app. The details of those trade-offs are in the System Design series rate limiting post.

Second: behind a proxy, RemoteIpAddress is the proxy's address, not the client's. I simulated two different clients with X-Forwarded-For:

== BehindProxy=false
200 200 200 200 200 | second client: 429
== BehindProxy=true
200 200 200 200 200 | second client: 200

With forwarded headers not processed, the first client filled the limit and the second client, which had not sent a single request, got a 429; the whole world looks like a single IP. Once UseForwardedHeaders was added, the second client got its own partition. That this line alone is not enough, and that which proxies to trust (KnownProxies, KnownIPNetworks) must be configured correctly, is covered in detail in the third post of the Web Servers and Hosting series. A misconfigured trust list means an attacker can bypass the IP limit entirely by writing a different X-Forwarded-For on every request.

Do not set limits by guesswork. First roll out the limiter with a very high value, log or emit a metric instead of returning 429, watch the real distribution for a week or two, then start with a limit a few times higher than your busiest legitimate client.

Common Pitfalls

Not setting RejectionStatusCode: clients see 503, and retry policies and load balancers treat it as a server error.

Calling UseRateLimiter before UseRouting. Endpoint-specific policies and DisableRateLimiting are read from endpoint metadata; if the middleware runs before an endpoint has been selected, only the global limiter applies. WebApplication adds routing at the very start, so with minimal APIs this is rarely a problem, but if you call UseRouting yourself, check the order.

Deriving the partition key from unvalidated input (a random API key, a random user name): every value is a new limiter, unbounded memory.

Limiting login only by IP or only by user name. One misses the distributed attack, the other misses the attack that tries many accounts from one IP; you need both.

A Small Experiment

★☆☆ Add a sliding window with SegmentsPerWindow = 2 to the algorithm comparison. How many requests of the second group get through, and why?

★★☆ Add a ConcurrencyLimiter as a third layer of the global chain (at most 50 concurrent requests for the whole system) and confirm through the log why the Retry-After header is missing on requests that ConcurrencyLimiter rejects.

★★★ Change LoginThrottle to reset the counter on a successful login. The RateLimiter API does not allow a reset; solve it by adding a "generation" counter to the partition key or by writing your own counter, and compare the memory behavior of the two approaches.

Rate limiting capped the request volume. In the next post we turn to the response itself: emitting security headers and a nonce-based CSP (Content Security Policy) from ASP.NET Core.