Receiving Webhooks: Signature Verification, Replay Protection, Fast 2xx and Idempotent Processing

Version note: The code in this post was built and run with .NET 10, Microsoft.Extensions.Http.Resilience 10.11, Polly 8.9, WireMock.Net 2.31 and xUnit v3 4.0.

Guess First

Bookstore, the online bookstore used as the shared example across this site's series, is the receiving side in this post: when a payment completes, the payment provider sends Bookstore a payment.succeeded event. Three questions:

First: your endpoint binds the body to a record, and you compute the signature by turning that record back into a string with JsonSerializer.Serialize. You use the same secret as the provider. Does the signature match?

Second: the provider sent the same event twice. What should you return for the second one: 409, 422 or 200?

Third: when the event arrives, the endpoint confirms the order, issues the invoice and sends an e-mail; 30 seconds in total. The provider's timeout is 10 seconds. What happens?

The answer to the first is "no", and the reason is a single character: in the test below the provider's +00:00 comes out of System.Text.Json's default encoder as \u002B00:00. The answer to the second is 200; otherwise the provider keeps sending. In the third case the provider gives up at second 10 and sends the event again, and you start processing the same payment a second time.

In the previous post we were the side sending webhooks and listed what we expect from receivers: verify the signature, filter repeats by webhook-id, answer with a fast 2xx. In this post we write that receiver.

Reading the Raw Body Before Model Binding

The signature is computed over the bytes the provider sent. The only reliable way to get those bytes is to read the body without turning it into anything. That is why the endpoint takes no DTO, only HttpRequest:

// Anonymous by design: the signature, not a user, proves who is calling
app.MapPost("/webhooks/payment-provider", PaymentWebhookEndpoint.ReceiveAsync);

// Verify -> store in the inbox -> queue -> 200. No business logic before the response
public static async Task<IResult> ReceiveAsync(
    HttpRequest request,
    PaymentWebhookInbox inbox,
    IOptions<PaymentWebhookOptions> options,
    TimeProvider time,
    ILogger<PaymentWebhookInbox> logger,
    CancellationToken cancellationToken)
{
    // The raw bytes, exactly as sent: the signature covers these, not a re-serialized object
    var body = await ReadBodyAsync(request, options.Value.MaxBodyBytes, cancellationToken);
    if (body is null)
    {
        return Results.StatusCode(StatusCodes.Status413PayloadTooLarge);
    }

    var id = request.Headers["webhook-id"].ToString();
    var verification = WebhookSignature.Verify(id, request.Headers["webhook-timestamp"], request.Headers["webhook-signature"],
        body, options.Value.Secrets, time.GetUtcNow(), options.Value.Tolerance);
    if (verification != WebhookVerification.Valid)
    {
        logger.LogWarning("Rejected payment webhook {WebhookId}: {Reason}", id, verification);
        return Results.Unauthorized();
    }

    if (!inbox.TryAdd(new InboxMessage { Id = id, Body = body, ReceivedAt = time.GetUtcNow() }))
    {
        // A copy we already have: acknowledge it, otherwise the provider keeps resending
        logger.LogInformation("Payment webhook {WebhookId} is a duplicate, acknowledged", id);
        return Results.Ok();
    }

    logger.LogInformation("Payment webhook {WebhookId} stored in the inbox", id);
    return Results.Ok();
}

ReadBodyAsync reads the body in 8 KB chunks and stops with a 413 once it goes past 64 KB. We have to hold the whole body in memory before the signature is verified, which exposes the endpoint to large unsigned requests that fill memory. We check the limit against Content-Length first and then against the bytes actually read, because Content-Length can lie or be missing.

The trap from the guessing section, for real, is in a test. The provider's JSON has spaces after colons and commas and the timestamp ends with +00:00; it is perfectly valid JSON. The test turns the same body around with JsonSerializer.Deserialize and Serialize and computes the signature over the original body:

original:     {"type": "payment.succeeded", "timestamp": "2026-11-14T10:00:00.000000+00:00", "data": {"id": "ch_000001", "status": "succeeded", "amount": 54890, "currency": "TRY", "merchant_reference": "bookstore-1001-01a1d981c28477d7827fbdd49814cbb4"}}
reserialized: {"type":"payment.succeeded","timestamp":"2026-11-14T10:00:00.000000\u002B00:00","data":{"id":"ch_000001","status":"succeeded","amount":54890,"currency":"TRY","merchant_reference":"bookstore-1001-01a1d981c28477d7827fbdd49814cbb4"}}
status: 401

The spaces are gone and the + became \u002B. System.Text.Json's default encoder escapes some characters for safety; as the output shows, + is one of them. The two strings carry the same data but different bytes, and HMAC looks at bytes. The Standard Webhooks specification also calls parsing the body and serializing it again a very common mistake on the receiving side.

Verifying the Signature: Constant-Time Comparison and Rotation

Verification repeats the sender's computation and compares the results. We added Verify to the WebhookSignature class from the previous post:

// Receiving side: any "v1," signature in the header may match any of our active secrets (rotation)
public static WebhookVerification Verify(string? messageId, string? timestampHeader, string? signatureHeader,
    ReadOnlySpan<byte> body, IEnumerable<string> secrets, DateTimeOffset now, TimeSpan tolerance)
{
    if (string.IsNullOrEmpty(messageId) || string.IsNullOrEmpty(signatureHeader)
        || !long.TryParse(timestampHeader, out var timestamp))
    {
        return WebhookVerification.MissingHeaders;
    }

    // Replay protection: a captured request is only usable within the tolerance window
    var sent = DateTimeOffset.FromUnixTimeSeconds(timestamp);
    if (sent < now - tolerance || sent > now + tolerance)
    {
        return WebhookVerification.StaleTimestamp;
    }

    foreach (var secret in secrets)
    {
        var expected = ComputeHmac(secret, messageId, timestamp, body);
        foreach (var candidate in signatureHeader.Split(' ', StringSplitOptions.RemoveEmptyEntries))
        {
            if (!candidate.StartsWith("v1,", StringComparison.Ordinal))
            {
                continue;
            }

            var provided = new byte[expected.Length];
            // Constant-time comparison: no early exit that leaks how many bytes matched
            if (Convert.TryFromBase64String(candidate[3..], provided, out var written)
                && written == expected.Length
                && CryptographicOperations.FixedTimeEquals(provided, expected))
            {
                return WebhookVerification.Valid;
            }
        }
    }

    return WebhookVerification.BadSignature;
}

CryptographicOperations.FixedTimeEquals compares two arrays all the way through without stopping at the first differing byte. SequenceEqual or a string comparison returns at the first difference; an attacker measuring response times can guess how many bytes of the signature are correct. The specification explicitly asks for constant-time comparison. We compare the decoded bytes, not the base64 strings, so different but equivalent base64 spellings don't cause trouble either.

The two loops are there for rotation. During a transition the sender can put more than one v1, signature in the header; on the receiving side we may be the ones changing the secret. Payments:Provider:Webhooks:Secrets is an array: the new secret first, the old one behind it during the transition. The value in appsettings.json is a made-up test secret for this sample; a real secret reaches configuration from a secret store. A test verifies that with two secrets configured, a request signed with the old one is accepted.

Replay Protection: Timestamp and webhook-id Together

Someone who captures a signed request can send it again as is; the signature is still valid. Two measures complement each other. webhook-timestamp is part of the signature, so it can't be changed, and we reject requests outside a 5-minute tolerance outright; in the test, a request signed 10 minutes ago gets 401. Repeats inside the window are caught by webhook-id: if the inbox already holds the id, the request is not processed. The tolerance must be wide enough to absorb clock differences between servers and narrow enough not to let a captured request be used for long; the official Standard Webhooks JavaScript library uses 5 minutes as well.

A Fast 2xx: Inbox and Queue

The sending side works with a short timeout; in the previous post Bookstore waited 15 seconds, our fake payment provider waits 10. If no answer arrives in that time, it sends the event again. If you try to confirm the order, issue the invoice and send the e-mail inside the endpoint, two things happen: on a slow day the provider sends the same event again and again, and during an outage every event piles up in the provider's retry queue.

The rule is simple: verify, store durably, enqueue, return 200. The real work happens in the background. PaymentWebhookInbox stores events keyed by webhook-id and writes the id to a Channel<string>:

// webhook-id is the key: the second copy of an event is recognized here, before any processing
// In memory in this sample; in Bookstore a WebhookInbox table with the id as primary key
public bool TryAdd(InboxMessage message)
{
    lock (_messages)
    {
        if (!_messages.TryAdd(message.Id, message))
        {
            return false;
        }
    }

    _queue.Writer.TryWrite(message.Id);
    return true;
}

In the real application the inbox is a table with webhook-id as its primary key. When the second copy arrives, the insert hits the unique constraint and the endpoint returns 200. Because the row is durable before the response, the event is not lost even if the process crashes right after returning 200; a step that re-queues rows left in Received at startup completes the picture (one of the exercises at the end). The Inbox pattern for message consumers is covered in the Idempotent Consumer and Inbox post of the Working with Message Queues series; here is its webhook version.

I measured the response time in a test. The first request got 200 in 3 ms; the copy with the same id also got 200 but was not processed:

passed Bookstore.Api.Tests.PaymentWebhookTests.Signed_webhook_is_acknowledged_fast_and_processed_once (1s 932ms)
    first=200 in 3 ms, copy=200, inbox=Processed attempts=1, order=Paid

Returning 409 or 422 for a copy is wrong: the provider counts every non-2xx response as a failed delivery and retries. From the provider's point of view, the second copy was delivered successfully too.

Idempotent Processing and Event Order

PaymentWebhookProcessor is a BackgroundService that reads ids from the channel and processes the inbox row. Every step is written so it can run again:

var attempt = attempts.FindByReference(evt.Data.MerchantReference);
if (attempt is null)
{
    throw new InvalidOperationException($"No payment attempt for reference {evt.Data.MerchantReference}.");
}

if (attempt.Status == PaymentAttemptStatus.Succeeded)
{
    // Checkout, a retry or reconciliation got there first: nothing to do
    logger.LogInformation("Order {OrderId} is already paid, webhook {WebhookId} changes nothing", attempt.OrderId, message.Id);
    return;
}

// The signature says who sent it, not that it is right: ask the provider for the current state
await using var scope = scopes.CreateAsyncScope();
var gateway = scope.ServiceProvider.GetRequiredService<IPaymentGateway>();
var charge = await gateway.FindChargeAsync(attempt.Reference, cancellationToken);
if (charge?.ProviderPaymentId != evt.Data.Id || evt.Data.Amount != (long)(attempt.Amount * 100))
{
    throw new InvalidOperationException($"Webhook {message.Id} does not match the provider's charge for {attempt.Reference}.");
}

Two decisions here. First, we don't trust the data in the webhook blindly. The signature proves the event came from the provider; but an integration bug can send a wrong reference, or the provider account may be compromised. So we compare the amount with our own attempt and confirm the charge through the provider's API. The security side of not trusting third-party responses is covered in the API Security series, in the post on APIs that talk to the outside world.

Second, order. There is no guarantee webhooks arrive in the order they were sent; in the previous post we didn't give one either. For a single payment, payment.refunded may arrive first and a delayed payment.succeeded after it. Treating an event as a "something changed" signal rather than as "work to do" solves this: the processor asks the provider for the current state and decides based on that. Where you can't query, you can use the payload's timestamp or a version number to ignore events older than the last change in your own record. We also keep state transitions one-way: a Succeeded attempt never becomes Open again.

If processing fails, the row stays in the inbox as Failed with the error message. We already returned 200 to the provider, so it won't send again; reprocessing failed rows is our responsibility. For the payment itself there is a second safety net: the reconciliation job from post 06 already queries open attempts.

Closing the Unknown Outcome With a Webhook

At the end of post 06 we mentioned a third way: the provider telling us when the payment is done. I started the fake provider with a 15-second processing time and a webhook address; it deliberately sends the event twice:

dotnet run --project src/Bookstore.FakeProviders -- --payments Slow --payments-delay 15 --payments-webhook http://localhost:5180/webhooks/payment-provider --webhook-copies 2

Checkout gave up after 8 seconds and returned 202. The API log (with timestamps):

13:00:53.673 warn: Bookstore.Ordering.CheckoutService
      Order 1001: payment outcome unknown, key 01a1d980-9837-7b1b-80d6-23a7afec145a
13:01:00.929 info: Bookstore.Payments.Webhooks.PaymentWebhookInbox
      Payment webhook msg_01a1d980d2e579fc9128a4abe6013681 stored in the inbox
13:01:00.932 info: Bookstore.Payments.Webhooks.PaymentWebhookInbox
      Payment webhook msg_01a1d980d2e579fc9128a4abe6013681 is a duplicate, acknowledged
13:01:00.937 info: Bookstore.Payments.Webhooks.PaymentWebhookProcessor
      Order 1001 paid by webhook msg_01a1d980d2e579fc9128a4abe6013681 (ch_000001)

The provider reported the moment it finished, and the order became Paid within 8 milliseconds; no need to wait for the reconciliation job's 2-minute MinAge. The copy was recognized in the inbox and acknowledged without processing. We now have three ways to close an unknown outcome: the customer retrying with the same key, the webhook, and reconciliation. The first two are fast but not guaranteed; a webhook can get lost, a customer may not come back. Reconciliation is slow but always runs. Using all three together covers each one's weak side with the others.

Testing Webhooks During Local Development

A real provider's sandbox can't send requests to your localhost. In this series the provider is imitated by a fake built on WireMock.Net, so that isn't a problem, but when you want to test end to end against the provider's sandbox you need a tunnel that exposes your local port to the internet. On Windows, Microsoft's dev tunnels tool does this. The commands from Microsoft Learn:

winget install Microsoft.devtunnel
devtunnel user login
devtunnel host -p 5180 --allow-anonymous

The host command prints an address of the form https://<tunnel_id>-5180.<region>.devtunnels.ms/; that is what you enter as the webhook address in the provider's dashboard. The provider sending webhooks can't log in to the tunnel, so anonymous access is required, and as the documentation warns, anyone who knows the tunnel address can reach your local server. Keep the tunnel open only while testing; signature verification is on anyway, so unsigned requests get 401. These commands require signing in with a Microsoft or GitHub account, so I did not run them for this post; they come from the CLI reference on Microsoft Learn.

In Production

Keep the webhook endpoint outside authentication middleware, but not outside rate limiting. Unsigned requests get a cheap 401; still, nobody should be able to keep the endpoint busy with thousands of requests per second. Rate limiting at the gateway level is covered in the rate limiting post of the ASP.NET Core Security series.

Monitor the inbox. The number of rows waiting in Received and the age of the oldest one show whether the processor is stuck; Failed rows are a list for a person to look at.

Don't keep the secret in a configuration file. The value in the sample is a made-up test secret; in production it should come from a secret store, and during a rotation, decide how long the old secret stays in the array based on the provider's transition period.

If the provider publishes an IP list, you can use it as an extra check, but not instead of the signature. IPs change; the signature proves itself on every request.

Common Pitfalls

Verifying the signature against re-serialized JSON. A space, field order or a single escaped character breaks the signature; in the test above that character was +.

Doing the work inside the request. A 30-second operation exceeds the provider's timeout; the provider resends the event and you start the same work twice.

Returning an error for a duplicate event. The provider counts it as a failed delivery and keeps sending; recognize the copy and return 200.

Comparing signatures with ==. A comparison that exits early opens the door to a timing attack; use CryptographicOperations.FixedTimeEquals.

Trusting event order. A delayed event can overwrite a newer state; ask the provider for the current state or keep transitions one-way.

A Small Experiment

★☆☆ Start the fake provider with --webhook-copies 3. How many stored in the inbox lines and how many is a duplicate lines do you see in the API log?

★★☆ Lower PaymentWebhookOptions.Tolerance to 1 second and, in a test, move webhook-timestamp 2 seconds into the future. What does the endpoint return? What would happen in production if the servers' clocks were 3 seconds apart?

★★★ Write an IHostedService that re-queues inbox rows left in Received when the application starts. When you move the inbox from a dictionary to a table in SQL Server, what does webhook-id being the primary key give you when two API instances receive the same event at the same moment?

With this post both directions of Bookstore's conversation with the outside world are complete: outgoing calls are time-bounded, retryable, idempotent and observable; incoming webhooks are verified, protected against repeats and answered fast. The last post of the series looks back: moving from Microsoft.Extensions.Http.Polly to Http.Resilience, from Polly v7 to v8, and what .NET 11 brings to the HttpClient side.