Caching: In-Memory and Distributed Cache with Redis

With tests we made sure the API works correctly. Now we move to another question: does it work fast? As an application grows, pulling the same data from the database over and over becomes a serious cost. But most of this data doesn't change on every request. This is exactly the waste caching prevents. Today we talk about how to improve performance by temporarily storing frequently requested but rarely changing data.

What Is a Cache and Why Is It Needed?

A cache is temporarily keeping frequently used data in a place where you can reach it fastest. The idea is simple: if you obtained a piece of data once with an expensive operation (for example, a database query), set it aside. Next time the same data is requested, give it directly from that side, without going to the database at all. This way you both respond much faster and reduce the database's load.

Think of it like a cook's workstation. They don't go to the pantry and bring back the frequently used salt, pepper, and oil every time; they keep them at hand, on the counter. Going to the pantry (the database) is slow; the counter (the cache) is instantly accessible. They keep rarely used ingredients in the pantry but always keep near them the things they reach for a hundred times a day. This is exactly the caching logic: keep the frequently touched data on the nearest shelf.

What to Cache and What Not to Cache?

Not every piece of data is suitable for caching. A good cache candidate has two properties: it's frequently requested and rarely changes. For example, data like a country list, category names, or site settings are perfect candidates; everyone wants them but they almost never change. In contrast, constantly changing data whose every request needs to see the most up-to-date state (for example, a user's instant balance) are bad candidates for caching. Caching the wrong data harms rather than helps, by showing the user stale information.

In-Memory Cache: The Simplest Way

The simplest way to cache in .NET is to keep the data directly in the application's own memory. This is called in-memory cache. Its setup is very easy; first you register the service:

builder.Services.AddMemoryCache();

Then you ask for IMemoryCache inside a service and use it. The typical pattern is: first look at the cache, if it's there return it directly; if not, pull it from the database, put it in the cache, and return it that way:

public async Task> GetCountriesAsync()
{
    if (_cache.TryGetValue("countries", out List? cached))
    {
        return cached!;
    }

    var countries = await _context.Countries.ToListAsync();

    _cache.Set("countries", countries,
        TimeSpan.FromMinutes(30));

    return countries;
}

Follow the logic. First, with TryGetValue, we look at whether the "countries" key is in the cache. If it is, we return it directly without going to the database at all; this is where the speed comes from. If not, we pull it from the database, put it in the cache with Set, and tell it to stay there for a while (here 30 minutes). This duration is called the cache's lifetime; when it expires, the data drops from the cache and the next request refreshes it. The first request is a bit slow, but all subsequent requests are lightning-fast.

The Limit of In-Memory Cache

In-memory cache is great but has an important limitation. The data stays in that application instance's own memory. So what if your application runs not on a single server but on multiple servers? In the twelfth article, while discussing lifetimes, we touched on a similar distributed scenario. Each server has its own memory, and therefore its own separate cache. When one server caches a piece of data, the other doesn't know about it. The result: different results can be returned to the same request depending on which server answered it. This inconsistency is a serious problem in scaling applications.

Distributed Cache: Redis Comes In

The solution to this problem is to take the cache out of each server's own memory and move it to a single central place that all of them access in common. This is a distributed cache, and the most common tool for this job is Redis. Redis is a very fast in-memory data store; it's used as a central cache that all servers connect to and share data in common.

Let's extend the cook analogy from earlier. A single cook's own counter was like in-memory cache. But if there are many cooks in a big restaurant, having different ingredients on each of their separate counters creates confusion. Instead, you set up a central prep station that all of them access in common; the ingredients stay there, in a single and consistent place. Redis is that central station in the kitchen: all servers look at the same cache, everyone sees the same data.

Connecting Redis as a distributed cache in .NET is also just a service registration:

builder.Services.AddStackExchangeRedisCache(options =>
{
    options.Configuration =
        builder.Configuration.GetConnectionString("Redis");
});

We don't embed Redis's connection information in the code, as we learned in the fifteenth article, but read it from configuration. After registering this, you use the IDistributedCache interface in the code. The usage pattern is very similar to in-memory cache (first look, if not there pull and put) but now the cache lives not on a single server but in the central Redis shared by all servers. So no matter how many servers you have, they all see the same consistent cache.

The Hardest Problem of Caching: Consistency

Caching sounds great but has a cost, and this cost is often overlooked. The data you cached can diverge from the real data underneath it. Let's say you cached a category list, then someone changed one of those categories. The cache still holds the old state; users see the old data until the cache's lifetime expires. This is called the cache "going stale."

There are two basic solutions to this problem. The first is giving the cache a reasonable lifetime; if the data refreshes every 30 minutes, there's at most 30 minutes of staleness, which is acceptable for most data. The second is clearing the cache by hand when the data changes; for example, when a category is updated, you delete the relevant cache key so that the next request pulls fresh data. Which one you choose depends on how up-to-date the data needs to be. There's a famous saying in software: one of the two hardest things is invalidating the cache at the right time. So think from the start not just about adding the cache, but also about when to clear it.

A Small Experiment

Choose a piece of data that's frequently requested but rarely changes (for example, a category or country list) and add in-memory cache to the service that returns it. Make consecutive requests to the endpoint; observe (for example, with a log) that the database is gone to on the first request, and that subsequent requests return from the cache. Then keep the cache's lifetime short (for example, 10 seconds) and see how the data refreshes when the duration expires. If you like, go one step further: change the data but don't clear the cache, and notice that the user sees the old data for a while. This experiment shows both the power of caching and the consistency trap at the same time.

In the next article we'll move on to health checks and monitoring. While an application runs in production, how do we automatically monitor whether it's healthy and whether it can reach the database? We'll talk about the ways to make your API not just fast, but also up and observable.