Redis Hashes: Storing Objects, Field Expiration and Memory Efficiency

Version note: The examples in this post were run with Redis 8.10.3 (redis:8), Valkey 9.1.3, .NET 10 and StackExchange.Redis 3.4.35.

Guess First

The shopping cart of Bookstore, the online bookstore used as the shared example across this site's series, is moving to Redis. On average every customer's cart holds five books: an ISBN and a quantity. The team comes up with three proposals: one hash per cart (field = ISBN, value = quantity), one JSON string per cart, or a separate key for every cart line (bookstore:cart:7:9780441013593). Let's write 10,000 carts all three ways and compare used_memory:

await MeasureAsync("hash per cart", customerId =>
    db.HashSetAsync($"bookstore:cart:{customerId}", [.. Isbns.Select((isbn, i) => new HashEntry(isbn, i + 1))]));

await MeasureAsync("JSON string per cart", customerId =>
    db.StringSetAsync($"bookstore:cart:{customerId}",
        JsonSerializer.Serialize(Isbns.Select((isbn, i) => new { isbn, quantity = i + 1 }))));

await MeasureAsync("key per cart item", customerId =>
    Task.WhenAll(Isbns.Select((isbn, i) => db.StringSetAsync($"bookstore:cart:{customerId}:{isbn}", i + 1))));

Which one uses the least memory and which the most? JSON text is already compact, and a hash's field names are text too. Is the difference large?

dotnet run --project src/Bookstore.RedisLab -- cart-storage

hash per cart          keys=10,000  memory= 1.41 MB
JSON string per cart   keys=10,000  memory= 2.75 MB
key per cart item      keys=50,000  memory= 3.37 MB

The hash takes about half of what JSON does; separate keys take 2.4 times the hash. The hash's advantage comes from the listpack encoding it uses while small: fields and values sit back to back in a single block of memory, with integers stored compactly. In the JSON string, quotes, colons and repeated names such as "quantity" also take bytes. With separate keys, every key carries its own metadata. This post covers what you need to know when using hashes: the commands, mapping in .NET, the choice between a hash and JSON, per-field expiration, encoding thresholds and Redis 8.10's compact hashes.

Hash Commands

A hash stores field-value pairs under one key; it is the natural structure for an object's properties or a dictionary. Commands work on individual fields:

127.0.0.1:6379> HSET bookstore:cart:7 9780441013593 1 9780451524935 2
(integer) 2
127.0.0.1:6379> HGET bookstore:cart:7 9780451524935
"2"
127.0.0.1:6379> HMGET bookstore:cart:7 9780441013593 9780000000000
1) "1"
2) (nil)
127.0.0.1:6379> HINCRBY bookstore:cart:7 9780441013593 2
(integer) 3
127.0.0.1:6379> HGETALL bookstore:cart:7
1) "9780441013593"
2) "3"
3) "9780451524935"
4) "2"
127.0.0.1:6379> HLEN bookstore:cart:7
(integer) 2
127.0.0.1:6379> HDEL bookstore:cart:7 9780451524935
(integer) 1
127.0.0.1:6379> HEXISTS bookstore:cart:7 9780451524935
(integer) 0
127.0.0.1:6379> OBJECT ENCODING bookstore:cart:7
"listpack"

HSET returns the number of new fields it added (updating an existing field does not count). HINCRBY is the field-level counterpart of the INCR we saw in post 01: it increases the quantity in the cart atomically, without reading and writing it on the client. The result of HGETALL is a flat array; under RESP3 the client receives it as a map. Field order is not guaranteed; if you need ordering, look at the sorted set in post 06.

HashEntry in .NET and Bookstore's Cart

In StackExchange.Redis, hash fields travel as HashEntry values (a name and a value). Bookstore.Api's cart endpoints use two hashes: bookstore:cart:{customerId} holds a quantity per ISBN, and bookstore:cart:{customerId}:price-locks holds, in kuruş, the price of books whose campaign price has been locked. Adding to the cart uses HINCRBY; if the quantity drops to zero, the field is removed:

// Add (or remove with a negative quantity) copies of a book: HINCRBY, then refresh the cart's lifetime
app.MapPost("/carts/{customerId:int}/items", async (int customerId, AddToCartRequest request, IConnectionMultiplexer redis) =>
{
    IDatabase db = redis.GetDatabase();
    RedisKey cart = RedisKeys.Cart(customerId);
    long quantity = await db.HashIncrementAsync(cart, request.Isbn, request.Quantity);
    if (quantity <= 0)
    {
        await db.HashDeleteAsync(cart, request.Isbn);
    }
    await db.KeyExpireAsync(cart, CartLifetime); // sliding: every change gives the cart 30 more days
    return Results.Ok(new { request.Isbn, quantity = Math.Max(quantity, 0) });
});

The cart's lifetime is deliberately renewed on every write. In post 03 we called this a bug for a window counter; here it is exactly what we want: a cart nobody touches for 30 days counts as abandoned and disappears on its own. On the read side the two hashes and the cart's TTL are fetched in one round trip and turned into a record:

Task<HashEntry[]> itemsTask = db.HashGetAllAsync(cart);
Task<HashEntry[]> locksTask = db.HashGetAllAsync(locks); // expired fields are never returned
Task<TimeSpan?> ttlTask = db.KeyTimeToLiveAsync(cart);
await Task.WhenAll(itemsTask, locksTask, ttlTask);

Dictionary<string, long> lockedPrices = locksTask.Result.ToDictionary(e => (string)e.Name!, e => (long)e.Value);
RedisValue[] lockedIsbns = [.. lockedPrices.Keys.Select(k => (RedisValue)k)];
long[] lockTtls = lockedIsbns.Length == 0 ? [] : await db.HashFieldGetTimeToLiveAsync(locks, lockedIsbns); // HPTTL, milliseconds
Dictionary<string, double> secondsLeft = lockedIsbns.Zip(lockTtls).ToDictionary(p => (string)p.First!, p => p.Second / 1000.0);

CartItem[] items = [.. itemsTask.Result.Select(e => new CartItem(
    Isbn: e.Name!,
    Quantity: (int)e.Value,
    LockedPriceInKurus: lockedPrices.TryGetValue(e.Name!, out long price) ? price : null,
    PriceLockSecondsLeft: secondsLeft.TryGetValue(e.Name!, out double left) ? Math.Round(left) : null))];

curl -X POST http://localhost:5080/carts/7/items -H "Content-Type: application/json" -d '{"isbn":"9780441013593","quantity":1}'
{"isbn":"9780441013593","quantity":1}
curl -X POST http://localhost:5080/carts/7/items -H "Content-Type: application/json" -d '{"isbn":"9780441013593","quantity":2}'
{"isbn":"9780441013593","quantity":3}
curl -X POST http://localhost:5080/carts/7/items -H "Content-Type: application/json" -d '{"isbn":"9780141439518","quantity":1}'
{"isbn":"9780141439518","quantity":1}
curl -X POST http://localhost:5080/carts/7/price-locks -H "Content-Type: application/json" -d '{"isbn":"9780441013593","priceInKurus":29990}'
{"isbn":"9780441013593","expiresInSeconds":900}
curl http://localhost:5080/carts/7
{"customerId":7,"items":[{"isbn":"9780441013593","quantity":3,"lockedPriceInKurus":29990,"priceLockSecondsLeft":897},{"isbn":"9780141439518","quantity":1,"lockedPriceInKurus":null,"priceLockSecondsLeft":null}],"expiresInDays":30}

The mapping here is written by hand because the field names are data (ISBNs). If you store an object with fixed field names, such as a book's title, author and price, a small extension method that turns the properties into a HashEntry[] is enough; a general reflection-based mapper is rarely needed.

Hash or JSON String?

The memory difference in Guess First is not a reason to decide on its own. The real question is how you access the data:

• If you read and change single fields (adding a book to the cart, bumping a quantity), the hash wins clearly: HINCRBY is atomic, whereas with a JSON string every change means "read, deserialize, modify, serialize, write" and the lost update problem from post 03 comes back.
• If you need a lifetime per field (the price lock below), only the hash has it.
• If the data is nested (a book's list of authors, its dimensions, its formats), a hash stays flat; its values are strings. Squeezing a nested structure into a hash means flattening field names into things like dimensions.width, which quickly gets messy. In that case a JSON string, or the JSON type covered in post 11, is more natural.
• If you always read and write the object as a whole and never update it partially, a JSON string is simple and sufficient; that is why we kept cache entries as serialized strings in the Caching series.

Per-Field Expiration

Until Redis 7.4 a lifetime could only be set on a key. Version 7.4 brought the HEXPIRE, HPEXPIRE, HTTL and HPERSIST family, which sets lifetimes on hash fields; Redis 8.0 added HGETEX, HSETEX and HGETDEL, which set the lifetime together with a read or a write. In Bookstore we use them to lock a campaign price for 15 minutes:

127.0.0.1:6379> HSET bookstore:cart:7:prices 9780441013593 34990 9780451524935 19990
(integer) 2
127.0.0.1:6379> HEXPIRE bookstore:cart:7:prices 900 FIELDS 1 9780441013593
1) (integer) 1
127.0.0.1:6379> HTTL bookstore:cart:7:prices FIELDS 3 9780441013593 9780451524935 9780000000000
1) (integer) 900
2) (integer) -1
3) (integer) -2
127.0.0.1:6379> OBJECT ENCODING bookstore:cart:7:prices
"listpackex"
127.0.0.1:6379> HEXPIRE bookstore:cart:7:prices 900 NX FIELDS 1 9780441013593
1) (integer) 0
127.0.0.1:6379> HEXPIRE bookstore:cart:7:prices 0 FIELDS 1 9780451524935
1) (integer) 2
127.0.0.1:6379> HGETALL bookstore:cart:7:prices
1) "9780441013593"
2) "34990"
127.0.0.1:6379> HSETEX bookstore:cart:7:prices EX 900 FIELDS 2 9780441013593 34990 9780451524935 19990
(integer) 1
127.0.0.1:6379> HGETEX bookstore:cart:7:prices PERSIST FIELDS 1 9780451524935
1) "19990"
127.0.0.1:6379> HGETDEL bookstore:cart:7:prices FIELDS 1 9780451524935
1) "19990"

The return codes differ a little from the EXPIRE you are used to, and each field gets its own array element: 1 the lifetime was set, 0 the condition (NX, XX, GT, LT) was not met, 2 the field was deleted right away because the lifetime was zero or in the past, -2 no such field. In HTTL, -1 means no expiry and -2 means the field does not exist. Once a field with a lifetime was added, the encoding switched to listpackex: a listpack that also carries the fields' lifetimes.

On the .NET side the counterpart of HSETEX is HashFieldSetAndSetExpiryAsync; this is the whole price lock endpoint:

// Campaign price is held for 15 minutes per book: HSETEX with a per-field TTL
app.MapPost("/carts/{customerId:int}/price-locks", async (int customerId, LockPriceRequest request, IConnectionMultiplexer redis) =>
{
    await redis.GetDatabase().HashFieldSetAndSetExpiryAsync(
        RedisKeys.CartPriceLocks(customerId), request.Isbn, request.PriceInKurus, PriceLockLifetime);
    return Results.Ok(new { request.Isbn, expiresInSeconds = (int)PriceLockLifetime.TotalSeconds });
});

When the last field of a hash expires and is removed, the key goes too. I set the price lock's lifetime to zero with redis-cli and looked at the cart again:

127.0.0.1:6379> HEXPIRE bookstore:cart:7:price-locks 0 FIELDS 1 9780441013593
1) (integer) 2
127.0.0.1:6379> EXISTS bookstore:cart:7:price-locks
(integer) 0

curl http://localhost:5080/carts/7
{"customerId":7,"items":[{"isbn":"9780441013593","quantity":3,"lockedPriceInKurus":null,"priceLockSecondsLeft":null}],"expiresInDays":30}

Valkey 9.0 brought the same command family, and the session above returned the same replies on Valkey 9.1.3; the only difference was the encoding: when a field with a lifetime was added, Valkey converted the hash to hashtable, while Redis kept it as listpackex. In a system with many small hashes that have expiring fields, the two servers can therefore show different memory usage.

Encoding: listpack and hashtable

Small hashes live in a listpack: a single block of memory with fields and values back to back. Lookups are linear, but because the block is small it stays in the CPU cache and is fast. When a hash crosses either of two thresholds it is converted to a hashtable: hash-max-listpack-entries (number of fields) and hash-max-listpack-value (longest field or value in bytes). On Redis 8.10.3 and Valkey 9.1.3 they are 512 and 64. Let's cross the thresholds with real numbers:

dotnet run --project src/Bookstore.RedisLab -- hash-encoding

128 fields                 encoding=listpack   memory= 2,615 bytes
512 fields                 encoding=listpack   memory=10,295 bytes
513 fields                 encoding=hashtable  memory=35,485 bytes
1 field, value 64 bytes    encoding=listpack   memory=   128 bytes
1 field, value 65 bytes    encoding=hashtable  memory=   227 bytes

One extra field multiplied the memory by 3.4. A single 65-byte value also pushed a small hash into hashtable. The conversion is one-way: when I deleted 510 fields from a 513-field hash and left 3, the encoding was still hashtable. The Redis 8.6 release notes report a substantial memory reduction for hashtable-encoded hashes (by unifying the field name and value into a single structure); the numbers above are after that improvement. You can raise the thresholds, but since lookups in a listpack are linear, very large listpacks cost CPU. Start with the defaults and measure before changing them.

Redis 8.10's Compact Hashes and HIMPORT

If you keep every book in the catalogue as a hash, the field names repeat in every key: 100,000 books, title, author, price and stock 100,000 times. Redis 8.10's compact hashes remove that repetition: hashes that share a set of fields keep the field names in a shared template. For now you enter this encoding through the new HIMPORT command family: first you define a connection-local fieldset (HIMPORT PREPARE), then you send the values in the same order (HIMPORT SET). StackExchange.Redis wraps this in the HashImport type and sends the PREPARE itself, once per connection:

using HashImport bookFields = HashImport.Create(fields);
await Task.WhenAll(Enumerable.Range(1, 100_000).Select(id => db.HashImportAsync($"bookstore:book:{id}", bookFields,
    new RedisValue[] { $"Dune #{id}", "Frank Herbert", 34990, id % 40 })));

HSET     memory=12.88 MB
HIMPORT  memory= 9.85 MB, encoding=template-listpack

The same 100,000 books took 24 percent less memory. hash_templates and hash_template_keys in INFO stats and used_memory_hash_templates in INFO memory arrived to monitor this structure. When I later added a different field to a templated hash (HSET … isbn …), Redis created a new template and the encoding stayed template-listpack. When does it matter? Where there are many small hashes with the same schema and memory cost is a real discussion. For a few thousand carts it is not worth the effort. The command is new and Valkey does not have it; follow the release notes for details such as the hash-rdb-load-*-template-* settings that control conversion during RDB loading before you rely on it.

In Production

Keep the size of a hash bounded. Structures that naturally stay small, such as a cart, are fine; but hashes that grow without limit, such as "the view count of every book in one hash", one day reach a million fields. On such a hash, HGETALL alone makes the main thread wait; I measured it on a hash with a million fields:

HGETALL: 1,000,000 fields in 291 ms
HSCAN:   1,000,000 fields in 540 ms
slowlog: HGETALL 44.2 ms

HGETALL ran for 44 ms on the server, and nothing else ran during that time. HSCAN (HashScanAsync in .NET) took longer overall, but because it moves page by page it never showed up in SLOWLOG. Deleting a big key is just as expensive; use UNLINK, which frees memory in the background, instead of DEL. If you use per-field TTLs, remember that expiring fields are also removed by the active expire cycle and that they change the encoding. If you also run on Valkey, do not write code that depends on HIMPORT.

Common Pitfalls

HGETALL on a huge hash: a million fields locked the server for 44 ms. HSCAN, HMGET or splitting the hash are safer.

Splitting fields into separate keys: for 10,000 carts memory went up 2.4 times; on top of that, deleting a whole cart or setting its lifetime now means managing 5 keys together, which is harder to do atomically.

Mixing up a hash and a JSON string: keeping a frequently, partially updated object in a JSON string and updating it on the client brings back the lost update problem from post 03.

Not noticing that a long value changed the encoding: a single value longer than 64 bytes, such as a description, turns a small hash into a hashtable, and it does not go back.

Reading the return value of HEXPIRE like that of EXPIRE: 2 does not mean "lifetime set" but "field deleted".

A Small Experiment

★☆☆ Run the cart-storage experiment against Valkey (BOOKSTORE_REDIS=localhost:6380). Is the ranking of the three approaches the same?

★★☆ Imagine adding a note per book to the cart: write a note:{isbn} field to the same hash with a 100-character note. What happened to OBJECT ENCODING and MEMORY USAGE? How does total memory change if you move the notes to a separate hash?

★★★ Repeat the compact-hash experiment with books that have 12 fields instead of 4. Does the gain from HIMPORT grow with the number of fields? Then add a new field to one of the hashes with HSET and watch hash_templates in INFO stats.

In the next post we move on to lists and sets: a simple and a reliable queue, "the last 10 books viewed", book tags with intersection queries, and the daily unique visitor count.