Hashes, HMAC and Key Derivation: SHA-2, SHA-3, HMAC, HKDF and When to Use Which

Version note: The code in this post was built and run with .NET 10 and xUnit v3 4.0; the outputs were taken on Ubuntu 24.04 (OpenSSL 3.0).

Guess First

When a payment completes, Bookstore's payment provider sends an event to the POST /webhooks/payments endpoint. We got a secret from the provider's dashboard, and the provider adds a signature computed with that secret to every request. Someone on the team wrote the verification like this:

// First version: three separate bugs
string expected = Convert.ToHexString(SHA256.HashData(Encoding.UTF8.GetBytes(secret + json)));
bool valid = expected == request.Headers["Payment-Signature"];

The tests pass and the provider's events are accepted. How many problems do you see in these two lines? The first is the construction: putting the secret next to the message and hashing it is not a MAC. The second is the comparison: comparing hex strings with == leaks timing and behaves inconsistently on upper versus lower case. The third is the input: the json variable is most likely the body after it was deserialized and produced again, so not the bytes the provider signed. In this post we fix these three problems in order, then move on to deriving purpose-specific keys from a single master key, that is, HKDF.

The Three Things a Hash Guarantees

A cryptographic hash function turns input of any length into a fixed-length digest and promises three properties. Preimage resistance: finding the input from the digest is practically impossible. Second preimage resistance: given an input, finding another input with the same digest is practically impossible. Collision resistance: finding any two inputs with the same digest is practically impossible. In practice, the first explains why a password hash can't be reversed, the second why a downloaded file matching a published checksum means something, and the third why signature algorithms can safely sign the digest instead of the data.

Think of Bookstore's nightly catalog export: a 50,000-line CSV file that goes to distribution partners through a file server. We put the file's SHA-256 digest next to it. We hash the large file through a stream without loading it into memory:

await using (FileStream stream = File.OpenRead(path))
{
    byte[] digest = await SHA256.HashDataAsync(stream);
    Console.WriteLine($"{stream.Length} bytes      sha256={Convert.ToHexStringLower(digest)}");
}

// Change one price on the first line: 101.00 -> 111.00
byte[] bytes = await File.ReadAllBytesAsync(path);
bytes[bytes.AsSpan().IndexOf("101.00"u8) + 1] = (byte)'1';
Console.WriteLine($"one byte changed  sha256={Convert.ToHexStringLower(SHA256.HashData(bytes))}");

$ dotnet run --project src/Bookstore.CryptoLab -- catalog-checksum
1051394 bytes      sha256=78c70e6eb00750f9fd515742d0bbb9f213352db6c64b6fa4291014d32613b1d0
one byte changed  sha256=7ad2ec316a8e3c72c70ed5311afc1a57e67c18818bc9afd566aad7459e327703

One byte changed in a one-megabyte file and the whole digest changed. If you run this command on Windows, the file is 1,101,394 bytes and the digest is different too: WriteAllLinesAsync uses Environment.NewLine as the line ending, which is \r\n on Windows. A hash digests bytes, not text; we'll come back to that detail at the end of the post.

It also needs saying clearly what a checksum doesn't protect. Someone who gets hold of the file can change a price, compute the new digest and replace both together; a hash has no key, anyone can compute it. A checksum catches accidental corruption, not deliberate changes. Against deliberate changes you either publish the digest through a separate, trusted channel or use a MAC or a signature.

SHA-2 or SHA-3?

The SHA-2 family (SHA-256, SHA-384, SHA-512) is the default choice today; it's on every platform, hardware acceleration is common and it has no known practical weakness. SHA-3, standardized in 2015, is built on a completely different internal structure, the sponge construction; it stands ready as a fallback should a structural problem ever be found in SHA-2. The SHA3_256, SHA3_384 and SHA3_512 classes that came with .NET 8 depend on the platform, as we saw in the first post of the series: on Linux they need OpenSSL 1.1.1 or later, on Windows build 25324 of Windows 11 or later, and they don't exist on macOS.

The practical rule: unless a protocol or the other party asks for SHA-3, use SHA-256. If you need SHA-3, put the call behind a SHA3_256.IsSupported check and decide at startup what happens on a platform that doesn't support it. Also store which algorithm you used alongside the data; being able to read old records if you ever have to switch depends on it. We'll give this idea, crypto agility, plenty of room in the post-quantum post of the series.

Why Not SHA256(secret + message): Length Extension and HMAC

SHA-256 and SHA-512 use the Merkle–Damgård structure, which processes the input in blocks and keeps an internal state after each block. The output is the internal state after the last block itself. The consequence: someone who knows SHA256(secret + message) can compute a valid digest for the message with bytes of their choosing appended, without ever knowing the secret. This class of attack is called length extension. We won't go into the details; what you need to know is that "put the secret first and hash" is not a safe MAC with SHA-2. SHA-3 and the truncated variants (SHA-384, SHA-512/256) don't have this property, but the right fix isn't picking a different hash, it's using the construction designed for the job.

That construction is HMAC. Defined in 1997 in RFC 2104, HMAC mixes the key with two different constants and computes a two-stage hash; because the inner hash's output is wrapped again by the outer hash, length extension doesn't work. In .NET it's one line:

byte[] mac = HMACSHA256.HashData(secret, message);

The rule for key length is simple: at least the hash output length, so at least 32 random bytes for HMAC-SHA256. A shorter key doesn't break HMAC but lowers its security to the key's length. Don't use a human-chosen value such as a password directly as an HMAC key; as we'll see below, passwords are a world of their own.

Verifying the Payment Webhook with HMAC

The webhook endpoint itself, that is, reading the body raw, preventing reprocessing by event id and rejecting an event whose signature is right but whose content is inconsistent, was covered in the API Security series' post on webhooks and outbound calls; signed machine-to-machine requests are in the same series' machine-to-machine authentication post. Here we focus on the crypto side: the signed bytes, the MAC computation, the comparison and secret rotation. Bookstore, the online bookstore used as the shared example across this site's series, keeps this code in the Webhooks folder of the Bookstore.Crypto library.

The format agreed with the provider: a Unix timestamp and one or more v1 signatures in the header; the signed bytes are the timestamp, a dot and the body itself:

// Header format: "t=<unix seconds>,v1=<hex HMAC-SHA256>[,v1=<hex>...]"
// Signed bytes: "<t>." followed by the raw request body, exactly as received
public static class PaymentWebhookSignature
{
    public static string CreateHeader(ReadOnlySpan<byte> secret, long timestamp, ReadOnlySpan<byte> body)
    {
        byte[] mac = ComputeMac(secret, timestamp, body);
        return $"t={timestamp},v1={Convert.ToHexStringLower(mac)}";
    }

    internal static byte[] ComputeMac(ReadOnlySpan<byte> secret, long timestamp, ReadOnlySpan<byte> body)
    {
        byte[] prefix = Encoding.ASCII.GetBytes(timestamp.ToString(CultureInfo.InvariantCulture) + ".");
        byte[] signed = new byte[prefix.Length + body.Length];
        prefix.CopyTo(signed, 0);
        body.CopyTo(signed.AsSpan(prefix.Length));
        return HMACSHA256.HashData(secret, signed);
    }
}

Putting the timestamp inside the signature matters: if the timestamp lived only in the header, an old request could be resent with a new timestamp. The verifying side:

public sealed class PaymentWebhookVerifier(IReadOnlyList<byte[]> secrets, TimeProvider time, TimeSpan tolerance)
{
    public WebhookVerdict Verify(string? header, ReadOnlySpan<byte> rawBody)
    {
        if (!TryParse(header, out long timestamp, out List<byte[]> candidates))
        {
            return WebhookVerdict.MalformedHeader;
        }

        DateTimeOffset sentAt = DateTimeOffset.FromUnixTimeSeconds(timestamp);
        if ((time.GetUtcNow() - sentAt).Duration() > tolerance)
        {
            return WebhookVerdict.StaleTimestamp;
        }

        // Current secret and, during rotation, the previous one
        foreach (byte[] secret in secrets)
        {
            byte[] expected = PaymentWebhookSignature.ComputeMac(secret, timestamp, rawBody);
            foreach (byte[] candidate in candidates)
            {
                if (Tags.AreEqual(expected, candidate))
                {
                    return WebhookVerdict.Valid;
                }
            }
        }

        return WebhookVerdict.BadSignature;
    }
}

There are four decisions. We convert the hex values in the header to bytes and accept only exactly 32-byte ones as candidates; a truncated signature is dropped while parsing. We measure the time difference as an absolute value with Duration(), so a timestamp from the future is rejected too. The comparison is Tags.AreEqual from the first post, that is, FixedTimeEquals. Finally, we accept more than one secret: when you renew the secret in the provider's dashboard, there's a window between the moment the provider switches to the new secret and your deploy. During rotation the old secret stays in the list, and it's removed afterwards.

Taking TimeProvider in the constructor lets the tests pin the clock. The lab command also plays the provider and tries five cases. The secrets here are obviously fake, readable values such as dev-only-payment-webhook-secret-v2, so you get the same signature on your machine:

$ dotnet run --project src/Bookstore.CryptoLab -- webhook
Payment-Signature: t=1794648600,v1=50f7edb849549930fcea2bd4d83b1d251cb61d3854f4cb5d375f03d7a60a8ce4
as sent            : Valid
amount changed     : BadSignature
10 minutes old     : StaleTimestamp
previous secret    : Valid
truncated v1       : MalformedHeader

One of the tests shows the third problem from Guess First directly. When the same data is produced again with a single space after a colon, the signature no longer matches:

[Fact]
public void Re_serialized_json_is_not_the_signed_body()
{
    // Same data, different bytes: a space after the colon
    byte[] reformatted = Encoding.UTF8.GetBytes(Encoding.UTF8.GetString(Body).Replace("\"amount\":", "\"amount\": "));
    Assert.Equal(WebhookVerdict.BadSignature, Verifier(Current).Verify(Header(Current, Now), reformatted));
}

That's why the endpoint first reads the body as bytes and verifies the signature, and only then deserializes. Serializing the object produced by model binding again and verifying that breaks at the first opportunity because of field order, whitespace or number formatting. The problem of deciding which bytes are signed is called canonicalization, and it comes back in the fifth post when we design our own signed format.

$ dotnet test
Test run summary: Passed!
  total: 18
  failed: 0
  succeeded: 18
  skipped: 0
  duration: 1s 315ms

Purpose-Specific Keys from One Master Key with HKDF

Bookstore needs more than one key: in the next post, one key to encrypt the Customer.Phone field, a separate blind index key to search on that field, and a separate key per institution to sign the notification webhooks sent to institutional customers. Generating and storing a separate secret for each is possible, but it gets harder to manage as the number of institutions grows. The alternative is to keep one strong master key in a safe place and derive the other keys from it.

The tool designed for this is HKDF, defined in RFC 5869. It has two steps. Extract turns the input keying material, the raw key where derivation starts, into a uniformly random intermediate key, with an optional salt. Expand produces output of the requested length from that intermediate key, depending on the info parameter. info is the parameter doing the real work here: keys derived from the same master key with different info values are independent of each other.

public static class KeyPurposes
{
    public const string CustomerPhoneEncryption = "bookstore/customer-phone/encryption/v1";
    public const string CustomerPhoneBlindIndex = "bookstore/customer-phone/blind-index/v1";

    public static string InstitutionWebhook(Guid institutionId) =>
        $"bookstore/institution-webhook/{institutionId:N}/v1";
}

// Derives purpose-bound subkeys from one master key (HKDF, RFC 5869)
public sealed class KeyDerivation
{
    private readonly byte[] _masterKey;

    public KeyDerivation(ReadOnlySpan<byte> masterKey)
    {
        if (masterKey.Length < 32)
        {
            throw new ArgumentException("The master key must be at least 256 bits.", nameof(masterKey));
        }

        _masterKey = masterKey.ToArray();
    }

    public byte[] DeriveKey(string purpose, int length = 32)
    {
        ArgumentException.ThrowIfNullOrWhiteSpace(purpose);
        return HKDF.DeriveKey(HashAlgorithmName.SHA256, _masterKey, length, info: Encoding.UTF8.GetBytes(purpose));
    }
}

$ dotnet run --project src/Bookstore.CryptoLab -- derive-keys
72d54ad3aac6ab20...  bookstore/customer-phone/encryption/v1
72d54ad3aac6ab20...  bookstore/customer-phone/encryption/v1
28b0ea64c3d3131e...  bookstore/customer-phone/blind-index/v1
2be7072ef45f76c4...  bookstore/institution-webhook/0197f00100007000800000000000a001/v1
1ea4910b4a274bc7...  bookstore/institution-webhook/0197f00100007000800000000000a002/v1

The same purpose always gives the same key, so the derived key doesn't need to be stored anywhere; it's recomputed when needed. Different purposes and different institutions get unrelated keys. The lab's master key is the bytes of the text dev-only-bookstore-master-key-32; an obviously fake value chosen so the outputs are the same for everyone. In production the master key is 32 random bytes and lives in a secret store; how to store and distribute it is the subject of the Configuration and Secret Management series.

Let's explain why we don't pass a salt. RFC 5869 says a salt strengthens HKDF significantly, doesn't have to be secret and can be reused; it also says the extract step can be skipped when the input is already a strong key. Our master key is already 32 uniformly random bytes; using it without a salt is safe here. In the fourth post, when we turn the shared secret produced by ECDH into a key, things are different: that value isn't uniformly random, so the extract step and a salt are a must there.

The /v1 suffix in info isn't there for nothing. If you ever need to change the derivation scheme, say the length or the hash, keys derived with v2 won't mix with the v1 ones and you can keep reading old data. In the API Security series' machine-to-machine authentication post we derived each API key's signing secret with HMACSHA256(master, "signing:" + keyId); that's a simplified form of HKDF's expand step. Using HKDF directly in new code is both standard and handles details such as length and salt correctly for you.

Passwords Are a Different World: PBKDF2, Argon2id and KMAC

HKDF is a fast function, which is the right property for strong keys. Passwords need the opposite: a password's entropy is low, so an attacker tries candidate passwords one by one and we want every attempt to be expensive. That's why password-based key derivation functions are deliberately slow: PBKDF2 raises the cost with an iteration count, Argon2id additionally with memory. Using PBKDF2 in .NET (Rfc2898DeriveBytes.Pbkdf2), iteration counts and ASP.NET Core Identity's PasswordHasher were covered in the "OWASP A04: Cryptographic Failures (2)" post of the Web Security (OWASP Top 10) series.

The key note from there applies here too: OWASP's password storage guide puts Argon2id first, but .NET's own crypto library has no Argon2. You have to pick a third-party package, and the choice itself is a security decision. The main options on NuGet today: Konscious.Security.Cryptography.Argon2, written fully in managed code (latest version 1.3.1, June 2024), and NSec.Cryptography and Geralt, which wrap libsodium (both released in 2026, MIT licensed). The managed package is easy to install, but its performance and maintenance rest with a single project; the libsodium wrappers rely on a well-reviewed native library but carry a native binary for every target platform. Whichever you choose, store the parameters (memory, iterations, parallelism) with the hash and include the package in dependency scanning.

One last note on KMAC, which came with .NET 9: it's HMAC's counterpart in the SHA-3 world and additionally takes a customization string. You can use that parameter to produce different MACs with the same key in different contexts:

byte[] a = Kmac256.HashData(key, data, outputLength: 32, customizationString: "Bookstore.Catalog"u8);
byte[] b = Kmac256.HashData(key, data, outputLength: 32, customizationString: "Bookstore.Ordering"u8);

$ dotnet run --project src/Bookstore.CryptoLab -- kmac
Catalog  : b399bfaeb6ce4c64d05cb643f9d7dfdd...
Ordering : e8ae5f9734dc9ae833e732482840cd1b...

KMAC depends on the platform: it needs OpenSSL 3.0 or later on Linux and build 26016 of Windows 11 or later on Windows, and isn't available on macOS. That's why the command checks Kmac256.IsSupported first. Unless the other side specifically asks for KMAC, HMAC-SHA256 is the safer default that works everywhere.

Production Notes

Log failed webhook verifications with their reason: BadSignature and StaleTimestamp say very different things. A sudden rise in StaleTimestamp is usually not an attack but the server's clock drifting; monitor NTP synchronization. A rise in BadSignature means either a half-finished secret rotation or someone probing the endpoint. Write the verdict and the event id to the log, not the signature or the secret.

Write secret rotation down as a procedure: add the new secret to the list and deploy, switch to the new secret in the provider's dashboard, remove the old one from the list a few days later. A verifier that accepts more than one secret makes this procedure seamless; with a verifier that accepts a single secret, rotation always means a short outage.

Keys derived from the master key are as valuable as the master key: if the master key leaks, every derived key counts as leaked. So rotating the master key includes rotating the derived keys. In the next post we make that rotation manageable by writing the key version into the encrypted data.

On the performance side, HMAC-SHA256 and HKDF are cheap; thousands of calls per second are no problem. You could cache a derived key briefly instead of recomputing it on every request, but the cache itself holds key material; don't let it sit in memory longer than needed.

Common Pitfalls

The first pitfall is thinking a hash is encryption. "We encrypted the phone number with SHA-256" is wrong twice: a hash can't be reversed, so it isn't encryption; but for a small value space like phone numbers, hashing every possibility and comparing takes a few minutes, so it doesn't give confidentiality either. Confidentiality needs encryption and search needs a keyed blind index; both are the subject of the next post.

The second is using one key for more than one purpose. Using the same secret both to sign webhooks and to encrypt fields carries a weakness in one use over to the other and makes rotation impossible. Derive a separate key for each purpose with HKDF.

The third is not thinking about encoding when turning a string into bytes. Hashes and MACs work on bytes; two texts that look the same can turn into different bytes:

string composed = "Café";          // é as one code point (NFC)
string decomposed = "Café";       // e + combining accent (NFD)
Console.WriteLine($"equal as string (ordinal): {string.Equals(composed, decomposed, StringComparison.Ordinal)}");
Console.WriteLine($"UTF-8  NFC : {Hash(Encoding.UTF8.GetBytes(composed))}");
Console.WriteLine($"UTF-8  NFD : {Hash(Encoding.UTF8.GetBytes(decomposed))}");
Console.WriteLine($"UTF-16 NFC : {Hash(Encoding.Unicode.GetBytes(composed))}");
Console.WriteLine($"UTF-8  NFD normalized to NFC: {Hash(Encoding.UTF8.GetBytes(decomposed.Normalize(NormalizationForm.FormC)))}");

$ dotnet run --project src/Bookstore.CryptoLab -- encoding
equal as string (ordinal): False
UTF-8  NFC : 73473dcc12b76308
UTF-8  NFD : c42cc7a1ca08364b
UTF-16 NFC : e3cb7a9fbd017ef0
UTF-8  NFD normalized to NFC: 73473dcc12b76308

Both show as "Café" on screen. File names coming from macOS and input from some keyboards can be in NFD. The rule has two parts: always pass the encoding explicitly as Encoding.UTF8 instead of relying on Encoding.Default or Encoding.Unicode, and bring human input into a single form with Normalize(NormalizationForm.FormC) before hashing it. For the webhook the opposite holds: there we don't touch the text at all and use the incoming bytes as they are.

The fourth is comparing a MAC as a hex string or truncating it to the incoming signature's length; we saw both in the first post. Convert the incoming value to bytes, check its length, compare with FixedTimeEquals.

A Small Experiment

★☆☆ Lower the tolerance of PaymentWebhookVerifier from 5 minutes to 30 seconds and run the tests. Which test broke? Think about when such a narrow window would reject legitimate requests in real life.

★★☆ Write an InstitutionWebhookSigner for the notification webhooks sent to institutional customers: it should derive the key with KeyPurposes.InstitutionWebhook(institutionId) and produce the header in the same format as PaymentWebhookSignature. Add a test showing that signatures produced for two different institutions are rejected by each other's verifier.

★★★ Add a v2 scheme to KeyPurposes: a version that derives 64 bytes with SHA-512 for the same purpose. Design a type that stops code which doesn't know which scheme a key was derived with from mixing up v1 and v2 (for example DerivedKey(string Purpose, int Version, byte[] Material)).

In the next post we use the bookstore/customer-phone/encryption/v1 key we derived with HKDF: authenticated encryption with AES-GCM and ChaCha20-Poly1305, nonce management, binding the encrypted value to the customer record with associated data, envelope encryption, and encrypting the Customer.Phone field with an EF Core value converter.