Redis Strings and Expiration: SET Options, Counters, Compare-and-Set and TTL Semantics

Version note: The examples in this post were run with Redis 8.10.3 (redis:8), Valkey 9.1.3, .NET 10 and StackExchange.Redis 3.4.35.

Guess First

When a new customer signs up at Bookstore, the online bookstore used as the shared example across this site's series, they receive a 6-digit verification code by e-mail. The code lives in Redis for 5 minutes. The customer cannot find the e-mail and presses "resend code"; the code behind that button writes the new code to the same key but forgets to pass the lifetime again:

127.0.0.1:6379> SET bookstore:verify:7 482913 EX 300
OK
127.0.0.1:6379> TTL bookstore:verify:7
(integer) 300
127.0.0.1:6379> SET bookstore:verify:7 105577
OK
127.0.0.1:6379> TTL bookstore:verify:7

What does the last TTL return? What is left of the old lifetime (about 300), or something else?

(integer) -1

-1 means "the key exists but has no expiry". The new code will never be deleted. SET does not just change the value, it rewrites the key and removes the existing TTL. The rule in the Redis documentation: commands that delete or overwrite the contents of a key (DEL, SET, GETSET, the *STORE commands) clear the timeout; commands that change the value in place (INCR, LPUSH, HSET) leave it alone. The right fix is to write the code with EX every time; for cases where you want to keep the current lifetime there is KEEPTTL:

127.0.0.1:6379> SET bookstore:verify:7 482913 EX 300
OK
127.0.0.1:6379> SET bookstore:verify:7 105577 KEEPTTL
OK
127.0.0.1:6379> TTL bookstore:verify:7
(integer) 300

This post covers Redis's most basic type, the string: the options of SET, counters, the compare-and-set commands that arrived with Redis 8.4, and how expiration works inside. We add three features to Bookstore: a daily view counter, a one-time verification code and an idempotency key for orders.

The Options of SET

A Redis string is a binary-safe byte sequence; it can be text, a number, a serialized object or an image. A single value can be at most 512 MB (proto-max-bulk-len defaults to 536,870,912 bytes), but post 01 showed what large values cost. Over the years SET has gained options, and today it does the job of several separate commands on its own:

127.0.0.1:6379> SET bookstore:lock:report token-a NX PX 30000
OK
127.0.0.1:6379> SET bookstore:lock:report token-b NX PX 30000
(nil)
127.0.0.1:6379> SET bookstore:book:42:price 349.90 XX
(nil)
127.0.0.1:6379> SET bookstore:book:42:title Dune GET
(nil)
127.0.0.1:6379> SET bookstore:book:42:title "Dune (Deluxe)" GET
"Dune"
127.0.0.1:6379> GETEX bookstore:book:42:title EX 60
"Dune (Deluxe)"
127.0.0.1:6379> GETDEL bookstore:verify:7
"105577"
127.0.0.1:6379> EXISTS bookstore:verify:7
(integer) 0

NX refused the second write: that is all there is to the command side of a distributed lock (why it is not enough on its own is covered in the Distributed Systems Fundamentals series). XX did not write because the key did not exist. The GET option returns the old value while writing; it replaced the old GETSET. Two related commands are handy when working with strings: GETEX changes the lifetime while reading the value (for sliding expiration), and GETDEL reads and deletes in one step. Redis 8.4's MSETEX writes several keys with the same lifetime in a single command; Valkey has had it since 9.1:

127.0.0.1:6379> MSETEX 2 bookstore:book:42:price 349.90 bookstore:book:1984:price 199.90 EX 600
(integer) 1
127.0.0.1:6379> TTL bookstore:book:1984:price
(integer) 600

Counters: INCR, INCRBY and INCRBYFLOAT

If a string holds a value that can be parsed as a signed 64-bit integer, Redis lets you use it as a counter. As we saw in post 01, these commands are atomic; increments arriving at the same time do not overwrite each other:

127.0.0.1:6379> SET bookstore:book:42:views 1250
OK
127.0.0.1:6379> INCR bookstore:book:42:views
(integer) 1251
127.0.0.1:6379> INCRBY bookstore:book:42:views 10
(integer) 1261
127.0.0.1:6379> DECR bookstore:book:42:views
(integer) 1260
127.0.0.1:6379> SET bookstore:book:42:rating 4.1
OK
127.0.0.1:6379> INCRBYFLOAT bookstore:book:42:rating 0.1
"4.2"
127.0.0.1:6379> INCR bookstore:book:42:title
(error) ERR value is not an integer or out of range
127.0.0.1:6379> SET bookstore:stock:9780441013593 9223372036854775807
OK
127.0.0.1:6379> INCR bookstore:stock:9780441013593
(error) ERR increment or decrement would overflow

Incrementing a non-numeric value and crossing the 64-bit limit both return errors; Redis does not overflow silently. INCRBYFLOAT adds fractions, but it stores the result as a string and is not suitable for precise calculations such as money; in Bookstore, prices are kept as integer amounts in the smallest currency unit or as decimal in the database.

In post 02 Bookstore's book view endpoint only incremented the total counter. Now we add a daily counter as well: bookstore:book:42:views:20261114. To keep daily keys from piling up forever, each one gets an 8-day lifetime, but only on the first increment. That is what EXPIRE … NX (Redis 7.0+) is for; in StackExchange.Redis it is ExpireWhen.HasNoExpiry:

app.MapPost("/books/{bookId:int}/views", async (int bookId, IConnectionMultiplexer redis, TimeProvider clock) =>
{
    IDatabase db = redis.GetDatabase();
    RedisKey daily = RedisKeys.BookViewsDaily(bookId, DateOnly.FromDateTime(clock.GetUtcNow().UtcDateTime));

    // Three commands, sent together (pipelined) and awaited together
    Task<long> total = db.StringIncrementAsync(RedisKeys.BookViews(bookId));
    Task<long> today = db.StringIncrementAsync(daily);
    Task<bool> ttl = db.KeyExpireAsync(daily, TimeSpan.FromDays(8), ExpireWhen.HasNoExpiry); // EXPIRE ... NX
    await Task.WhenAll(total, today, ttl);

    return Results.Ok(new { bookId, views = total.Result, today = today.Result });
});

curl -X POST http://localhost:5080/books/42/views
{"bookId":42,"views":1,"today":1}
curl -X POST http://localhost:5080/books/42/views
{"bookId":42,"views":2,"today":2}

docker compose -f lab/compose.yaml exec redis redis-cli TTL bookstore:book:42:views:20261114
(integer) 691200

691,200 seconds is 8 days; the second request did not reset it. The three commands travel in one round trip, but together they are not atomic: if the application crashes between INCR and EXPIRE, a daily key without a lifetime can be left behind. The next EXPIRE … NX gives that key its lifetime anyway, so as long as traffic continues during the day the problem fixes itself. Post 08 shows how to do this in one step (MULTI/EXEC, Lua and Redis 8.8's INCREX).

GET + SET: 9,617 Lost Increments

Let's see why atomic commands matter from the other side. This time we read the same counter on the client, add one and write it back; 50 concurrent callers, 200 times each:

Task[] callers = [.. Enumerable.Range(0, 50).Select(_ => Task.Run(async () =>
{
    for (int i = 0; i < 200; i++)
    {
        long current = (long)await db.StringGetAsync(key);
        await db.StringSetAsync(key, current + 1);
    }
}))];
await Task.WhenAll(callers);

Console.WriteLine($"expected 10000, got {await db.StringGetAsync(key)}");

expected 10000, got 383
expected 10000, got 382
expected 10000, got 383

9,617 of the 10,000 increments were lost. Each GET and each SET is atomic on its own, but the gap between them is not: the 50 callers read the same value and all write back the same number plus one. Multiplexing amplifies the effect, because 50 GETs go out back to back in the same pipeline and get the same reply. Whenever you see "read, compute, write" logic against Redis, the first question should be: can the server do this in one command? For a counter the answer is INCR. For "write only if the value is still what I expect" the answer is the compare-and-set below.

Compare-and-Set: SET IFEQ, DELEX and DIGEST

Before Redis 8.4, the way to make "delete if the value is X" or "change if the value is X" atomic was a Lua script. For years this script was used so that only the owner could release a lock; KEYS[1] is the lock key and ARGV[1] the token written when the lock was taken:

if redis.call("GET", KEYS[1]) == ARGV[1] then
    return redis.call("DEL", KEYS[1])
else
    return 0
end

Redis 8.4 moved this to the command level. SET gained the IFEQ, IFNE, IFDEQ and IFDNE conditions; DELEX was added for conditional deletes and DIGEST for hashing a value:

127.0.0.1:6379> SET bookstore:idem:7f3c9a processing NX EX 86400
OK
127.0.0.1:6379> SET bookstore:idem:7f3c9a order:1001 IFEQ processing KEEPTTL
OK
127.0.0.1:6379> SET bookstore:idem:7f3c9a order:1002 IFEQ processing KEEPTTL
(nil)
127.0.0.1:6379> GET bookstore:idem:7f3c9a
"order:1001"
127.0.0.1:6379> DIGEST bookstore:idem:7f3c9a
"967bf420e1a321ef"
127.0.0.1:6379> SET bookstore:verify:7 482913 EX 300
OK
127.0.0.1:6379> DELEX bookstore:verify:7 IFEQ 111111
(integer) 0
127.0.0.1:6379> DELEX bookstore:verify:7 IFEQ 482913
(integer) 1
127.0.0.1:6379> DELEX bookstore:verify:7 IFEQ 482913
(integer) 0

The second SET with IFEQ did not write, because the value was no longer processing. DELEX … IFEQ deletes only when the value matches and returns how many keys it deleted. DIGEST returns the XXH3 hash of the value in hex and is used with IFDEQ/IFDNE. When the value is large, say a JSON document of a few hundred KB, you send the 16-character digest for the comparison instead of the whole value. StackExchange.Redis exposes these conditions through the ValueCondition type: ValueCondition.Equal(...), NotEqual, DigestEqual; there is also a helper that computes the digest on the client.

The Valkey difference is clear here. Valkey 8.1 added SET … IFEQ and 9.0 brought DELIFEQ key value for conditional deletes; but IFNE, IFDEQ, IFDNE, DELEX and DIGEST do not exist. With docker compose -f lab/compose.yaml exec valkey valkey-cli:

127.0.0.1:6379> SET bookstore:lock:report token-a
OK
127.0.0.1:6379> DELIFEQ bookstore:lock:report token-b
(integer) 0
127.0.0.1:6379> DELIFEQ bookstore:lock:report token-a
(integer) 1
127.0.0.1:6379> DIGEST bookstore:idem:7f3c9a
(error) ERR unknown command 'DIGEST', with args beginning with: 'bookstore:idem:7f3c9a'

Bookstore: a One-Time Verification Code

Let's write the verification code from Guess First properly. Every time a code is created it is written with a 5-minute lifetime; verification is a single DELEX … IFEQ step: if the code is right it is deleted and success is returned, if it is wrong or expired nothing changes:

app.MapPost("/customers/{customerId:int}/verification-code",
    async (int customerId, IConnectionMultiplexer redis, ILoggerFactory loggers) =>
{
    string code = RandomNumberGenerator.GetInt32(100_000, 1_000_000).ToString();
    // A new code replaces the old one, together with a fresh 5-minute TTL
    await redis.GetDatabase().StringSetAsync(RedisKeys.VerificationCode(customerId), code, CodeLifetime);
    loggers.CreateLogger("Bookstore.Verification").LogInformation("Verification code for customer {CustomerId}: {Code} (would be e-mailed)", customerId, code);
    return Results.Accepted(value: new { expiresInSeconds = (int)CodeLifetime.TotalSeconds });
});

app.MapPost("/customers/{customerId:int}/verification-code/verify",
    async (int customerId, VerifyRequest request, IConnectionMultiplexer redis) =>
{
    // DELEX key IFEQ code: deletes only if the stored code matches; one atomic step (Redis 8.4+)
    bool verified = await redis.GetDatabase()
        .StringDeleteAsync(RedisKeys.VerificationCode(customerId), ValueCondition.Equal(request.Code));
    return verified
        ? Results.Ok(new { verified })
        : Results.BadRequest(new { verified, error = "invalid or expired code" });
});

curl -X POST http://localhost:5080/customers/7/verification-code
{"expiresInSeconds":300}

info: Bookstore.Verification[0]
      Verification code for customer 7: 713745 (would be e-mailed)

curl -s -w " %{http_code}" -X POST http://localhost:5080/customers/7/verification-code/verify -H "Content-Type: application/json" -d '{"code":"111111"}'
{"verified":false,"error":"invalid or expired code"} 400
curl -s -w " %{http_code}" -X POST http://localhost:5080/customers/7/verification-code/verify -H "Content-Type: application/json" -d '{"code":"713745"}'
{"verified":true} 200
curl -s -w " %{http_code}" -X POST http://localhost:5080/customers/7/verification-code/verify -H "Content-Type: application/json" -d '{"code":"713745"}'
{"verified":false,"error":"invalid or expired code"} 400

The same code cannot be used twice. Had we written it as a GET followed by a DEL, two concurrent requests with the same code could both get a match from GET and both succeed. GETDEL is an option too, but a wrong guess would delete the code and the user would have to ask for a new one because of a typo. One gap is left on purpose: the number of attempts is unlimited and a 6-digit code can be guessed. The verify endpoint needs rate limiting; the design is in "Rate Limiting in Depth" in the ASP.NET Core Security series.

Pointed at Valkey, the same API returned 500 from the verify endpoint: StackExchange.Redis.RedisServerException: ERR unknown command 'DELEX'. If you need to support Valkey, write this step with DELIFEQ (through ExecuteAsync) or with a Lua script.

Bookstore: an Idempotency Key for Orders

To keep a double click on the checkout page or a client retry from creating the same order twice, the client adds an Idempotency-Key header to every order. The design side is in the "Idempotency Keys: Not Creating the Same Order Twice on Double Clicks and Retries" post of the Solution Patterns series; here we build the Redis side. The first request takes the key with the value processing using NX, and once the order exists it records the result with IFEQ processing KEEPTTL:

IDatabase db = redis.GetDatabase();
RedisKey key = RedisKeys.Idempotency(idempotencyKey);

// SET key processing NX EX 86400: only the first request gets through
bool first = await db.StringSetAsync(key, Processing, IdempotencyWindow, When.NotExists);
if (!first)
{
    RedisValue stored = await db.StringGetAsync(key);
    return stored == Processing
        ? Results.Conflict(new { error = "a request with this key is still being processed" })
        : Results.Ok(new { orderId = (long)stored, replayed = true });
}

try
{
    long orderId = await db.StringIncrementAsync(RedisKeys.OrderIdSequence); // stands in for the real order insert
    // SET key <orderId> IFEQ processing KEEPTTL: record the result only if we still own the key
    await db.StringSetAsync(key, orderId, Expiration.KeepTtl, ValueCondition.Equal(Processing));
    return Results.Created($"/orders/{orderId}", new { orderId, replayed = false });
}
catch
{
    // Release the key so that the client can retry; only if it is still ours
    await db.StringDeleteAsync(key, ValueCondition.Equal(Processing));
    throw;
}

curl -s -w " %{http_code}" -X POST http://localhost:5080/orders -H "Content-Type: application/json" -H "Idempotency-Key: 7f3c9a" -d '{"customerId":7,"isbn":"9780441013593","quantity":1}'
{"orderId":1,"replayed":false} 201
curl -s -w " %{http_code}" -X POST http://localhost:5080/orders -H "Content-Type: application/json" -H "Idempotency-Key: 7f3c9a" -d '{"customerId":7,"isbn":"9780441013593","quantity":1}'
{"orderId":1,"replayed":true} 200
curl -s -w " %{http_code}" -X POST http://localhost:5080/orders -H "Content-Type: application/json" -H "Idempotency-Key: 8b21d0" -d '{"customerId":7,"isbn":"9780441013593","quantity":1}'
{"orderId":2,"replayed":false} 201
curl -s -w " %{http_code}" -X POST http://localhost:5080/orders -H "Content-Type: application/json" -H "Idempotency-Key: c0ffee" -d '{"customerId":7,"isbn":"9780441013593","quantity":1}'
{"error":"a request with this key is still being processed"} 409

The second request did not create a new order; it returned the result of the first. Before the last request I set the key bookstore:idem:c0ffee to processing by hand with redis-cli, imitating a request that is still in flight; for such a key the API returns 409. Thanks to KEEPTTL the 24-hour window is not reset when the result is written. The IFEQ condition matters too: if the key expired and was taken again by another request, the late first request does not overwrite it. In a real application the order is written to SQL Server, and the process can crash between those two steps; the key then stays at processing for 24 hours. The remedy is to give the processing state a short lifetime (say 30 seconds) and extend it when writing the result, or to keep the durable record in the database; the inbox table in "Idempotent Consumers and the Inbox: When the Same Message Arrives Twice" in the Working with Message Queues series is that second route.

How Expiration Works Inside

Redis does not store a key's lifetime as remaining seconds but as an absolute Unix timestamp in milliseconds. EXPIRETIME and PEXPIRETIME show it directly:

127.0.0.1:6379> SET k v
OK
127.0.0.1:6379> EXPIRE k 100
(integer) 1
127.0.0.1:6379> EXPIRETIME k
(integer) 1794652280
127.0.0.1:6379> PEXPIRETIME k
(integer) 1794652279939
127.0.0.1:6379> TTL nokey
(integer) -2
127.0.0.1:6379> PERSIST k
(integer) 1
127.0.0.1:6379> TTL k
(integer) -1

-2 means the key does not exist, -1 that it has no expiry. The absolute timestamp has two consequences. First, time keeps flowing while Redis is down: when you load an RDB file, keys whose time has passed are deleted during loading. Second, the lifetime depends on the machine's clock. The Redis documentation says so explicitly: if you move a running server's clock forward, keys whose TTL has not run out can be deleted immediately, and moving an RDB file between two machines whose clocks differ a lot can produce surprising results. That is why keeping the server clock correct with NTP matters for Redis too.

Expired keys are removed in two ways. In the passive (lazy) way, when a client tries to access a key, Redis checks its lifetime, deletes it if it has passed and answers as if the key did not exist. But some keys are never read again; for those the active way runs: Redis periodically samples keys with an expiry at random and deletes the expired ones. I measured the effect by writing 200,000 verification codes with lifetimes of 1–2 seconds and then never reading them:

t= 0.0s  DBSIZE=200,000  expired_keys=      0
t= 1.0s  DBSIZE=199,997  expired_keys=      3
t= 2.0s  DBSIZE=    181  expired_keys=199,954
t= 3.0s  DBSIZE=      0  expired_keys=200,000

Without any client reading them, all the keys left memory shortly after their lifetimes ran out. How often active cleanup runs and how much CPU it spends is governed by hz (default 10) and active-expire-effort (default 1); in systems where many keys expire at once, they are watched together with expired_keys and expired_stale_perc in INFO stats.

Replication follows a different rule: replicas do not delete keys on their own. When a lifetime runs out on the primary, the primary synthesises a DEL and sends it both to the replicas and to the AOF file. That way the decision to delete is made in one place. If a replica becomes primary, it starts expiring keys itself. Replication details are in post 13.

In Production

Decide deliberately what the TTL should be on every write path: a new lifetime (EX), the old one (KEEPTTL), or none. For strings beyond caching too, an accumulation of keys without expiry is a memory leak; watch the ratio of keys to expires in INFO keyspace. For counters where loss is acceptable, use FireAndForget (post 02) and pipelining; where it is not, await the result. If you use the compare-and-set commands, your minimum server version is 8.4; on a managed service or Valkey, check first which commands are supported. Every feature that relies on expiration also relies on the server clock being correct.

Common Pitfalls

Losing the TTL on overwrite: the SET in Guess First clears the lifetime, and the verification code stays valid forever.

Read-modify-write with GET + SET: 9,617 of 10,000 increments were lost. Use INCR, SET … IFEQ or the atomic tools in post 08.

EXPIRE on every increment of a window counter: if EXPIRE 60 is called after every INCR for a 60-second window, the window never closes. I compared both approaches for a customer sending a request every two seconds:

t=0s  always: count=1 ttl=60   NX: count=1 ttl=60
t=2s  always: count=2 ttl=60   NX: count=2 ttl=58
t=4s  always: count=3 ttl=60   NX: count=3 ttl=56
t=6s  always: count=4 ttl=60   NX: count=4 ttl=54

The counter that gets EXPIRE every time always jumps back to 60; as long as traffic does not stop it never resets, and after a while the customer hits the limit unfairly. EXPIRE … NX sets the lifetime only the first time.

Assuming Redis 8.4 commands on Valkey: DELEX and DIGEST do not exist, and of the SET conditions only IFEQ does.

Doing money arithmetic with INCRBYFLOAT: the result is limited by floating-point precision; keep amounts as integers.

A Small Experiment

★☆☆ Replace the GET + SET pair in the lost-update experiment with StringIncrementAsync and run it again. Then try the original version with a single caller instead of 50: why is nothing lost this time?

★★☆ Change the verify endpoint to compare by digest instead of by value: use ValueCondition.DigestEqual when verifying, not when storing the code. Check which command reaches Redis with INFO commandstats rather than SLOWLOG.

★★★ Give the processing state of the idempotency flow a 30-second lifetime and extend it to 24 hours when writing the result. Can you do that in a single command (SET … IFEQ … EX)? Does the same command work on Valkey?

In the next post we move on to hashes: storing an object split into fields, the per-field expiration commands from Redis 7.4 and 8.0, the memory difference between listpack and hashtable, and Bookstore's shopping cart.