Azure App Configuration: Centralized Configuration, Labels for Environments, Sentinel-Key Refresh and Key Vault References

Version note: The examples in this post were run with .NET 10 (SDK 10.0.402), Microsoft.Azure.AppConfiguration.AspNetCore 8.6.1 and the Azure App Configuration emulator 1.2.0 on Docker 29.2. No App Configuration store was created in Azure; the az commands were written from the examples on Microsoft Learn and were not run.

Guess First

Bookstore, the online bookstore used as the shared example across this site's series, reads its free shipping threshold from Azure App Configuration in staging. The refresh interval is five seconds. You change the threshold from 800 to 950 and wait a minute. Which value does the API return? And another one: what happens if you lower the refresh interval to 200 milliseconds so that every instance sees the new value on every request?

When Do You Need App Configuration, and When Not?

In post two we passed environment-specific settings as environment variables. That works well for a single app, but it has two limits. First, a change needs a restart or a new rollout: App Service restarts the app when an app setting changes, and in Kubernetes an environment variable change means new pods. Second, shared settings get copied: if Bookstore's API, the Ordering worker and the job that calculates campaigns all need the shipping threshold, the same value lives in three places and sooner or later one is forgotten.

Azure App Configuration is a central key-value store that solves both problems. Values live in one place, apps read them through a configuration provider and can refresh them without a deploy. Microsoft's FAQ positions App Configuration and Key Vault as complementary: secrets stay in Key Vault, and App Configuration holds only references to them.

When it is unnecessary is just as clear: if you have a single app, settings change once a month and every change ships with a deploy anyway, App Configuration only adds another dependency. Another service to reach at startup, another quota to watch, another resource to grant access to. Bookstore uses it because several services read the same business rules in the institutional order and campaign flows, and the business wants to change those rules without waiting for a deploy.

Key-Values, Labels and Environments

Every entry in App Configuration consists of a key, a value and an optional label. Keys are usually hierarchical, separated by :. A label distinguishes different versions of the same key; its most common use is the environment. Bookstore's settings for staging and production live in a file in the repo:

{
  "items": [
    { "key": "Bookstore:Shipping:FreeShippingThreshold", "value": "500", "label": null, "content_type": null, "tags": {} },
    { "key": "Bookstore:Shipping:StandardFee", "value": "49.90", "label": null, "content_type": null, "tags": {} },
    { "key": "Bookstore:Shipping:FreeShippingThreshold", "value": "800", "label": "Staging", "content_type": null, "tags": {} },
    { "key": "Bookstore:Sentinel", "value": "1", "label": "Staging", "content_type": null, "tags": {} },
    { "key": "Bookstore:Sentinel", "value": "1", "label": "Production", "content_type": null, "tags": {} }
  ]
}

Unlabeled entries are the shared default for every environment; the Staging label overrides only what differs. The file is in the Azure CLI's appconfig/kvset import format, so a pipeline can load it into the store with a single command. Separating environments by label has one limit: access is granted at store level. Microsoft's FAQ recommends a separate store per environment when environments need different permissions, and labels when that separation is not needed. Bookstore's production has its own store, while dev and staging share one store with labels.

A snapshot is an immutable copy of the key-values selected at a point in time. If you pin a release to a specific snapshot, later changes in the store do not affect it, and rolling back is as easy as selecting the previous snapshot. Dynamic refresh for frequently changing business rules and snapshots for settings that should be frozen with a release is a good split.

Wiring the Provider: Select, Label and TrimKeyPrefix

Bookstore's App Configuration setup lives in a separate extension method:

public static IConfigurationBuilder AddBookstoreAppConfiguration(
    this IConfigurationBuilder builder,
    IConfiguration configuration,
    IHostEnvironment environment,
    TokenCredential credential)
{
    var label = environment.EnvironmentName;
    var refreshInterval = configuration.GetValue("AppConfig:RefreshInterval", TimeSpan.FromSeconds(30));

    return builder.AddAzureAppConfiguration(options =>
    {
        // Endpoint + managed identity in Azure; a connection string for the local emulator
        if (configuration["AppConfig:ConnectionString"] is { Length: > 0 } connectionString)
        {
            options.Connect(connectionString);
        }
        else
        {
            options.Connect(new Uri(configuration["AppConfig:Endpoint"]!), credential);
        }

        options
            // Shared unlabeled values first, then the environment's label: the later one wins
            .Select(KeyPrefix + "*", LabelFilter.Null)
            .Select(KeyPrefix + "*", label)
            .TrimKeyPrefix(KeyPrefix)
            // Only the sentinel is watched; when it changes, all settings are reloaded
            .ConfigureRefresh(refresh => refresh
                .Register(SentinelKey, label, refreshAll: true)
                .SetRefreshInterval(refreshInterval))
            // Key Vault references are resolved with the app's own identity
            .ConfigureKeyVault(keyVault => keyVault.SetCredential(credential));
    });
}

The order of the Select calls is the layering itself: for the same key, the later selection wins. TrimKeyPrefix turns Bookstore:Shipping:FreeShippingThreshold into Shipping:FreeShippingThreshold, so the ShippingOptions class and validators from post two work unchanged. If other apps use the same store, each app selecting its own prefix keeps it from loading theirs.

For the credential I moved the code from post three into a separate class called AzureCredentials; App Configuration and Key Vault now use the same credential instance. In Azure, the app's managed identity gets the "App Configuration Data Reader" role on the store. In Program.cs the provider is added after the environment variables and before Key Vault:

var credential = AzureCredentials.Create(builder.Environment, builder.Configuration);

// Non-secret, centrally managed settings: App Configuration
var useAppConfiguration = AppConfigurationExtensions.IsEnabled(builder.Configuration);
if (useAppConfiguration)
{
    builder.Configuration.AddBookstoreAppConfiguration(builder.Configuration, builder.Environment, credential);
    builder.Services.AddAzureAppConfiguration();
}

var app = builder.Build();

if (useAppConfiguration)
{
    // Requests trigger the refresh; the middleware belongs early in the pipeline
    app.UseAzureAppConfiguration();
}

Running Locally with the Emulator

Microsoft provides an official, open-source emulator for App Configuration. It supports anonymous authentication and access keys with HMAC. For the app to connect, I enabled HMAC with a key generated in the work folder; anonymous access is on only so the seed script can write:

$ docker run -d --name secretseries-appconfig -p 8483:8483 \
    -e Tenant__AnonymousAuthEnabled=true -e Authentication__Anonymous__AnonymousUserRole=Owner \
    -e Tenant__HmacSha256Enabled=true \
    -e Tenant__AccessKeys__0__Id=bookstore-local -e Tenant__AccessKeys__0__Name=Primary \
    -e "Tenant__AccessKeys__0__Secret=$ACCESS_KEY" -e Tenant__AccessKeys__0__ReadOnly=true \
    mcr.microsoft.com/azure-app-configuration/app-configuration-emulator:1.2.0

$ python3 deploy/appconfig/seed-emulator.py deploy/appconfig/bookstore.kvset.json
Bookstore:Shipping:FreeShippingThreshold None 500
Bookstore:Shipping:StandardFee None 49.90
Bookstore:Shipping:FreeShippingThreshold Staging 800
Bookstore:Sentinel Staging 1
Bookstore:Sentinel Production 1

I started the API as staging with the emulator's connection string. That connection string is a secret and exists only for the local emulator. In Azure the app connects with the endpoint and its managed identity, and disabling access key authentication on the store is a good default:

$ export ASPNETCORE_ENVIRONMENT=Staging
$ export AppConfig__ConnectionString="Endpoint=http://localhost:8483;Id=bookstore-local;Secret=$ACCESS_KEY"
$ export AppConfig__RefreshInterval=00:00:05
$ dotnet Bookstore.Api.dll --urls http://localhost:5094 &
$ curl -s localhost:5094/api/info
{"environment":"Staging","freeShippingThreshold":800,"jwtKeyFingerprint":"5c69bb74","secrets":{"ConnectionStrings:Bookstore":false,"Jwt:SigningKey":true,"Email:ApiKey":true,"Payments:Webhook:Secret":false}}

800 came from the staging label; it overrode the 750 in appsettings.Staging.json because the App Configuration provider is added after the files and environment variables.

Refreshing Without a Deploy Using a Sentinel Key

The answer to the first opening question: nothing changes. The provider watches only the key registered with Register, and that is the sentinel. I changed the threshold and waited, then incremented the sentinel's value:

09:25:00 "freeShippingThreshold":800
PUT Bookstore:Shipping:FreeShippingThreshold Staging 950
09:25:04 "freeShippingThreshold":800
09:25:09 "freeShippingThreshold":800
09:25:13 "freeShippingThreshold":800
PUT Bookstore:Sentinel Staging 2
09:25:17 "freeShippingThreshold":800
09:25:21 "freeShippingThreshold":950

info: Microsoft.Extensions.Configuration.AzureAppConfiguration.Refresh[0]
      Setting updated. Key:'Bookstore:Sentinel'

When the sentinel changed, refreshAll: true reloaded all selected settings and the next request saw the new value. This behavior is a feature: while you change several related settings one by one (such as the threshold and the shipping fee), the app never sees a half-applied state. You write all the changes and bump the sentinel last; the change is applied in one go.

It matters how the refresh is triggered. According to the dynamic configuration page on Microsoft Learn, the UseAzureAppConfiguration middleware runs on every request but only goes to the store when the refresh interval has elapsed; the check runs asynchronously and does not hold up the request. An app that receives no requests never refreshes, and the request that triggers the check usually still sees the old value. Those two rules explain why the first request after the sentinel change above still returned 800: either the refresh interval had not elapsed yet, or the request that triggered the check saw the old value. In a process that receives no HTTP requests, such as a background job, you have to trigger the refresh yourself through IConfigurationRefresherProvider. The code reading the value must also use IOptionsMonitor or IOptionsSnapshot; as we saw in post two, IOptions keeps the first value.

Newer versions of the provider also offer RegisterAll() as an alternative to the sentinel: it watches all selected key-values and reloads when any of them changes. It takes less discipline than a sentinel, but every change is applied immediately; you lose the ability to change related settings atomically.

Key Vault References

Instead of putting a secret in App Configuration, you store a reference pointing to the secret in Key Vault. A reference is a small JSON value stored with a special content type:

{
    "key": "Bookstore:Email:ApiKey",
    "label": "Staging",
    "content_type": "application/vnd.microsoft.appconfig.keyvaultref+json;charset=utf-8",
    "value": "{\"uri\":\"https://localhost:8443/secrets/BookstoreApi-Email--ApiKey\"}",
    ...
}

The key point: App Configuration does not resolve the reference; the provider inside the app does. When the provider sees a reference, it goes to Key Vault with the credential given in ConfigureKeyVault. So the assumption that anyone who can read App Configuration can also read the secret is wrong; reading the secret also requires the Secrets User role on Key Vault. The app's identity needs both roles: Data Reader on the store and Secrets User on the vault.

I also tried the case where a reference cannot be resolved: I put a reference pointing to an unreachable vault address into the emulator and restarted the API:

Unhandled exception. Microsoft.Extensions.Configuration.AzureAppConfiguration.KeyVaultReferenceException: Key vault error. ErrorCode:'' Key:'Bookstore:Email:ApiKey' Label:'Staging' Etag:'T18-JcEXeEcvIDC-dK6OOg' SecretIdentifier:'https://localhost:8443/secrets/BookstoreApi-Email--ApiKey'
 ---> System.AggregateException: Retry failed after 7 tries. ... (Connection refused (localhost:8443)) ...

After about 50 seconds of retries the app did not start. Unlike the App Service reference in post three, the value does not quietly stay as the reference text here; the provider throws. If the same error happens during a refresh, the app does not stop: the provider logs a warning and keeps going with the cached value.

Feature flags are also stored in App Configuration under a separate key prefix and loaded by the same provider; evaluating them is covered in the Feature Flags post of the Deployment Strategies series. From a storage point of view there is one difference: flags are not tied to the sentinel and have their own refresh.

Production Notes

Do the quota math. According to Microsoft's FAQ, a Standard tier store allows up to 30,000 requests per hour; the Free tier is limited to 1,000 requests per day, and Premium has no quota. Per the worked example on the same page, most frequent, unchanged requests such as sentinel checks do not count against the hourly quota; what consumes it is every instance reloading every setting after a change. For 50 instances and 1,000 settings, one sentinel change means 500 requests.

You cannot build an "always current on every request" system by shrinking the refresh interval. The provider explicitly prevents it; with an interval of 200 milliseconds the app did not start:

Unhandled exception. System.ArgumentOutOfRangeException: The refresh interval cannot be less than 1000 milliseconds. (Parameter 'refreshInterval')

The default of 30 seconds is enough for most business rules; a change takes that interval plus one request to take effect.

Manage changes like code. A change made in the portal shows up in the audit log but never goes through review. In Bookstore the settings live in bookstore.kvset.json in the repo; a change arrives as a pull request, and the pipeline loads it into the store and bumps the sentinel last. In Azure that step looks like this (not run):

# In the pipeline: load settings from the file, then bump the sentinel
az appconfig kv import --endpoint https://bookstore-appconfig-staging.azconfig.io --auth-mode login \
  --source file --path deploy/appconfig/bookstore.kvset.json --format json --profile appconfig/kvset --yes
az appconfig kv set --endpoint https://bookstore-appconfig-staging.azconfig.io --auth-mode login \
  --key Bookstore:Sentinel --label Staging --value "$BUILD_ID" --yes

Put validation in the pipeline as well: as we saw in post two, an invalid value arriving through reload does not stop the app but breaks every request that reads those options. A small test that runs the same [OptionsValidator] rules before the file is loaded into the store keeps a bad value from reaching production. Revision history is another safety net: according to the FAQ it is kept for 30 days on the Standard and Premium tiers and 7 days on Free and Developer.

Common Pitfalls

Setting up App Configuration for a single app and adding a needless startup dependency. Putting the secret itself into App Configuration instead of a reference. Forgetting to update the sentinel and losing hours on "the change didn't apply". Placing the refresh middleware behind another middleware in the pipeline that may stop requests from ever reaching it. Forgetting that background jobs receive no HTTP requests and therefore never refresh. Reading the value with IOptions. Treating labels as a security boundary; anyone with access to the store reads every label. Changing a production setting by hand in the portal and being surprised when the next pipeline import reverts it.

A Small Experiment

★☆☆ Run the App Configuration emulator in Docker and write one key twice, unlabeled and with the Staging label. Start the app first in the Production and then in the Staging environment and compare which value it reads.

★★☆ Compare the sentinel with RegisterAll(): change two related settings (threshold and fee) at different times and follow the logs to see whether the app observes an inconsistent state in between with each approach.

★★★ Write a script that loads settings from a file in the repo into the store and bumps the sentinel last. Add a step that checks the values with the options validators before loading, and show that the pipeline stops on an invalid value.

On the Azure side, secrets are in Key Vault and settings are in App Configuration. But not every team is on Azure, and not every secret is static. In the next post we set up HashiCorp Vault, a cloud-independent secret server, and its open-source fork OpenBao, to issue short-lived SQL Server logins for Bookstore that are regenerated every time the app starts.