Configuration and Secrets in Compose: environment, env_file, .env Interpolation, secrets and Profiles

Version note: The examples in this post were run with Docker Compose 5.7.1, Docker Engine 29.9.3, SQL Server 2025 (RTM-CU11), .NET 10, Redis 8 and Mailpit 1.33.

Guess First

The .env file in the project folder has the line STORE_TIMEZONE=Europe/Istanbul. Is there an environment variable called STORE_TIMEZONE inside the Bookstore.Api container? Second question: the API's env_file says Store__TimeZone=Europe/London, its environment says Store__TimeZone=Europe/Istanbul. Which time zone does /api/store/clock return? Third: you ran docker compose down, but could the Mailpit you started with --profile tools still be running?

Two Separate Mechanisms: .env and env_file

Because their names are similar, these are the two things most often confused, and they do completely different jobs.

.env is for Compose itself. While reading compose.yaml, Compose fills ${VAR} expressions first from the environment variables in the shell and, if it does not find them there, from the .env in the project folder. This is called interpolation and the result is written into the text of the file; a variable in .env reaches no container unless compose.yaml uses it. That is why the answer to the first question is no.

env_file, on the other hand, belongs to a service: every line in it becomes an environment variable of that container. We collected the non-secret settings of Bookstore.Api in config/api.env and committed it:

# Non-secret Bookstore.Api settings for the Compose environment (committed)
Store__Culture=tr-TR
Logging__LogLevel__Microsoft.AspNetCore=Warning

  api:
    build:
      context: .
      dockerfile: src/Bookstore.Api/Dockerfile
    env_file: config/api.env
    environment:
      Store__TimeZone: ${STORE_TIMEZONE:-Europe/Istanbul}

The line in environment combines the two: the value comes through interpolation from the shell or from .env, :- falls back to Europe/Istanbul when neither has it, and the result is given to the container as Store__TimeZone. The double underscore is ASP.NET Core's hierarchy separator in environment variables; Store__TimeZone is read as Store:TimeZone in the application. The order of configuration providers is covered in the ASP.NET Core Web API series post "Configuration and the Options Pattern in ASP.NET Core: Provider Order, User Secrets and Validation".

docker inspect composeseries-api-1 --format "{{range .Config.Env}}{{println .}}{{end}}" | grep -E "Store|STORE"

Store__Culture=tr-TR
Store__TimeZone=Europe/Istanbul

There is no STORE_TIMEZONE in the container, only Store__TimeZone filled with its value.

Who Overrides Whom?

The order in the Compose documentation, strongest first: docker compose run -e, environment with an interpolated value, plain environment, env_file, and finally ENV in the image. In interpolation, the shell overrides .env as well. We tried the second question: we added Store__TimeZone=Europe/London to config/api.env, then passed another value from the shell, then removed the line again.

echo "Store__TimeZone=Europe/London" >> config/api.env
docker compose up -d api
curl -s localhost:8080/api/store/clock

STORE_TIMEZONE=Europe/Berlin docker compose up -d api
curl -s localhost:8080/api/store/clock

{"zone":"Europe/Istanbul","localTime":"2026-11-14 16:43 +03:00"}
 Container composeseries-api-1 Recreate
 Container composeseries-api-1 Recreated
 Container composeseries-api-1 Started
{"zone":"Europe/Berlin","localTime":"2026-11-14 14:43 +01:00"}

London in env_file lost to the value in environment. What stands out is that the first up did nothing: Compose computes the container's final environment, and since the result did not change, it does not recreate the container. STORE_TIMEZONE from the shell overrode the one in .env, the result changed and the container was recreated. A real change in env_file (when we changed Store__Culture) triggered a recreate the same way.

This has a side effect: an old environment variable left in the shell on CI or on a server silently overrides the value in .env. If you are not sure where a setting comes from, docker compose config shows the resolved result.

Empty Variables and ${VAR:?}

In the previous post's file the passwords came in as ${MSSQL_SA_PASSWORD}. We removed .env temporarily and tried:

docker compose config --quiet; echo "exit code: $?"

WARN[0000] The "MSSQL_SA_PASSWORD" variable is not set. Defaulting to a blank string.
WARN[0000] The "BOOKSTORE_DB_PASSWORD" variable is not set. Defaulting to a blank string.
WARN[0000] The "MSSQL_SA_PASSWORD" variable is not set. Defaulting to a blank string.
WARN[0000] The "BOOKSTORE_DB_PASSWORD" variable is not set. Defaulting to a blank string.
exit code: 0

One warning per place of use, and exit code 0. After this warning, easily lost in a CI log, SQL Server tries to start with an empty password and reports a completely different error in its own log. To make the variable required, use :?:

      MSSQL_SA_PASSWORD: ${MSSQL_SA_PASSWORD:?set MSSQL_SA_PASSWORD in .env}

error while interpolating services.db.environment.MSSQL_SA_PASSWORD: required variable MSSQL_SA_PASSWORD is missing a value: set MSSQL_SA_PASSWORD in .env
exit code: 1

Compose stops without starting anything and the message says what is missing. In short: ${VAR:-default} uses the default when the variable is empty or unset, ${VAR:?message} fails when it is empty or unset; ${VAR-default} and ${VAR?message} without the colon only look at whether it is unset and accept an empty value.

Where Does a Password in an Environment Variable Leak?

Making it required does not change where the password lives. A password passed as an environment variable is written into the container's definition:

docker inspect composeseries-api-1 --format "{{range .Config.Env}}{{println .}}{{end}}" | grep -i password
docker inspect composeseries-db-1 --format "{{range .Config.Env}}{{println .}}{{end}}" | grep -i password

ConnectionStrings__Default=Server=db;Database=BookstoreDb;User Id=bookstore_app;Password=<app-password>;TrustServerCertificate=True
MSSQL_SA_PASSWORD=<sa-password>

Anyone with access to Docker, every CI log that shows docker compose config output and every process inside the container (/proc/1/environ) can see these values, not to mention a library that dumps the environment into the log on an error. In post 13 of the Docker series we saw the same problem with docker run and passed the password as a file. The Compose counterpart is secrets:.

secrets: Passing the Password as a File

secrets:
  sa-password:
    file: ./secrets/sa-password.txt
  db-app-password:
    file: ./secrets/db-app-password.txt
  api-connection-string:
    file: ./secrets/api-connection-string.txt

The top-level secrets defines a secret's source (a file, or an environment variable on the host with environment:), the secrets in a service defines which service sees which secret. A secret given to a service sits in the container as a file under /run/secrets/<name>. The secrets/ folder has been in both .gitignore and .dockerignore since the Docker series; we added a small script that creates the files:

#!/usr/bin/env bash
# Creates the local development secrets under secrets/ (git- and docker-ignored). Run once per machine.
set -euo pipefail
cd "$(dirname "$0")/../.."
mkdir -p secrets
new_password() { printf '%s' "Bk$(openssl rand -base64 18 | tr -d '/+=')!9"; }

[ -f secrets/sa-password.txt ] || new_password > secrets/sa-password.txt
[ -f secrets/db-app-password.txt ] || new_password > secrets/db-app-password.txt
printf 'Server=db;Database=BookstoreDb;User Id=bookstore_app;Password=%s;TrustServerCertificate=True' \
    "$(cat secrets/db-app-password.txt)" > secrets/api-connection-string.txt
# The containers run as other users (SQL Server 10001, Bookstore.Api 1654): the files must be readable for them
chmod 644 secrets/*.txt
ls -l secrets

The three services read them in three different ways. Since the Docker series, Bookstore.Api has read /run/secrets with the key-per-file configuration provider: the file name is the key, the content is the value. We mounted the secret so that it matches that name:

    secrets:
      # Key-per-file: /run/secrets/ConnectionStrings__Default becomes ConnectionStrings:Default
      - source: api-connection-string
        target: ConnectionStrings__Default

For SQL Server we found something interesting. The environment variable list on Microsoft Learn only has MSSQL_SA_PASSWORD, but the name MSSQL_SA_PASSWORD_FILE also appears in the sqlservr binary. We tried it: the 2025 image started with only MSSQL_SA_PASSWORD_FILE=/run/secrets/sa-password set, and the sa login worked with the password in the file. We use it knowing it is undocumented behavior; if it ever disappears, one line in the entrypoint that reads the file and exports MSSQL_SA_PASSWORD does the same job. The health check and migrate read the password from the file at the moment they need it:

  db:
    image: mcr.microsoft.com/mssql/server:2025-latest
    environment:
      ACCEPT_EULA: "Y"
      # Not in the documented variable list, but SQL Server 2025 reads the sa password from this file
      MSSQL_SA_PASSWORD_FILE: /run/secrets/sa-password
    secrets:
      - sa-password
    healthcheck:
      test: SQLCMDPASSWORD="$$(cat /run/secrets/sa-password)" /opt/mssql-tools18/bin/sqlcmd -C -S localhost -U sa -b -Q "SELECT 1" -o /dev/null
    ...

  migrate:
    image: mcr.microsoft.com/mssql/server:2025-latest
    entrypoint: ["/bin/bash", "-c"]
    command:
      - >-
        SQLCMDPASSWORD="$$(cat /run/secrets/sa-password)"
        BOOKSTORE_DB_PASSWORD="$$(cat /run/secrets/db-app-password)"
        exec /opt/mssql-tools18/bin/sqlcmd -C -S db -U sa -b
        -i /scripts/01-bookstore.sql -i /scripts/02-app-login.sql
    secrets:
      - sa-password
      - db-app-password

docker inspect composeseries-db-1 --format "{{range .Config.Env}}{{println .}}{{end}}" | grep MSSQL_SA
docker inspect composeseries-api-1 --format "{{range .Mounts}}{{.Type}} {{.Source}} -> {{.Destination}} rw={{.RW}}{{println}}{{end}}"

MSSQL_SA_PASSWORD_FILE=/run/secrets/sa-password
bind /home/user/bookstore/config/appsettings.Production.json -> /app/appsettings.Production.json rw=false
bind /home/user/bookstore/secrets/api-connection-string.txt -> /run/secrets/ConnectionStrings__Default rw=false

The container definition now only holds the path to the file. The second line also shows how a secret works outside Swarm: Compose mounts the file on the host as a read-only bind mount. There is no encryption or separate storage; the gain is that the password leaves the environment, inspect and the config output. That has two consequences. First, the file's permissions on the host apply as they are. On Linux (WSL 2 included), if you leave the file open only to yourself with chmod 600, the API running as uid 1654 cannot read it:

Unhandled exception. System.UnauthorizedAccessException: Access to the path '/run/secrets/ConnectionStrings__Default' is denied.
 ---> System.IO.IOException: Permission denied

That is why the script has chmod 644. Second, the uid, gid and mode settings of the long syntax are not applied outside Swarm, and Compose 5.7.1 now says so explicitly:

WARN[0000] service "api": secrets.api-connection-string.uid: uid is not supported outside Swarm mode and will be ignored
WARN[0000] service "api": secrets.api-connection-string.mode: mode is not supported outside Swarm mode and will be ignored

One more note: when we gave secrets: to the pre_start hook from post 2, Compose 5.7.1 did not mount the files (cat: /run/secrets/db-app-password: Is a directory); in labs/compose/02-pre-start.yaml we pass the secret files to the hook with a bind mount.

configs: appsettings without Changing the Image

configs is the non-secret counterpart of secrets: it mounts a file at any path you want in the container. Without rebuilding the image, we gave the API an appsettings.Production.json; the image runs in the Production environment, so ASP.NET Core reads that file on its own:

{
  "Logging": {
    "Console": {
      "FormatterName": "simple",
      "FormatterOptions": {
        "SingleLine": true,
        "TimestampFormat": "HH:mm:ss ",
        "UseUtcTimestamp": true
      }
    }
  }
}

    configs:
      - source: api-settings
        target: /app/appsettings.Production.json
...
configs:
  api-settings:
    file: ./config/appsettings.Production.json

api-1  | 13:43:17 info: Microsoft.Hosting.Lifetime[0] Hosting environment: Production
api-1  | 13:43:17 info: Bookstore.Api[0] Catalog source: SqlBookSource

Log lines are now single-line with UTC timestamps; docker compose logs got easier to read. Instead of keeping a file, for small settings you can also write the content straight into compose.yaml with the config's content: attribute. Which should be an environment variable and which a file? Environment for short, single values (Store__TimeZone); a config file for structured or long settings; a secret for everything that is secret.

Optional Services with Profiles

Some services are not needed by everyone every day: Redis for cache experiments, Mailpit for testing the e-mail flow (a test SMTP server that catches sent e-mails and shows them in a web UI), an sqlcmd for querying the database by hand. We marked them with profiles:

  # Optional services: docker compose --profile cache / --profile tools
  redis:
    image: redis:8-alpine
    profiles: [cache]
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
    restart: unless-stopped

  mailpit:
    image: axllent/mailpit:v1.33
    profiles: [tools]
    ports:
      - "127.0.0.1:8025:8025"

  # Ad-hoc queries, never started by up: docker compose run --rm sqlcmd -Q "SELECT COUNT(*) FROM dbo.Books"
  sqlcmd:
    image: mcr.microsoft.com/mssql/server:2025-latest
    profiles: [cli]
    entrypoint:
      - /bin/bash
      - -c
      - SQLCMDPASSWORD="$$(cat /run/secrets/sa-password)" exec /opt/mssql-tools18/bin/sqlcmd -C -S db -U sa -d BookstoreDb "$$@"
      - sqlcmd
    secrets:
      - sa-password
    depends_on:
      db:
        condition: service_healthy

Services without a profile are always active; those with one only when that profile is enabled. A profile is enabled with the --profile flag or the COMPOSE_PROFILES environment variable; when a service is targeted directly (docker compose run sqlcmd), its profile is enabled automatically:

docker compose config --services
COMPOSE_PROFILES=tools,cache docker compose up -d
docker compose run --rm sqlcmd -W -Q "SELECT TOP 2 Title FROM dbo.Books ORDER BY PriceTry DESC"

db
migrate
api
web
...
SERVICE   STATUS
api       Up 5 seconds (healthy)
db        Up 57 seconds (healthy)
mailpit   Up 56 seconds (healthy)
redis     Up 56 seconds (healthy)
web       Up Less than a second
Title
-----
Dune
İnce Memed

(2 rows affected)

On the first attempt we had put sqlcmd in the tools profile, and --profile tools up -d started it too: sqlcmd without arguments found nothing to read on stdin and went straight to Exited (0). Long-running services and one-off tools should not share a profile; a separate profile for tools (cli) causes no trouble when used with run. "$$@" in the entrypoint and the trailing sqlcmd carry the arguments given to run into the sqlcmd inside bash -c.

The answer to the third question comes from this experiment too: services started with a profile are not in the model of commands that do not enable that profile.

docker compose down

 Network composeseries_default Removing
 Network composeseries_default Resource is still in use 

down removed the default services, but Mailpit kept running and the network could not be removed. For a clean shutdown, use the same profiles: docker compose --profile tools down, or write COMPOSE_PROFILES into .env. Seq can be added to a profile the same way, with the container definition from the .NET Logging series post "Reading Structured Logs with Seq: Docker Setup, Queries, Signals and Alerts".

In Production

A file-based secret in Compose is not a security boundary but a leak prevention measure: the file sits on the host in plain text. On a server, keep the files owned by the deploy user, with permissions that let only the container users who need them read them; writing secrets from a vault or the CI's secret store at deploy time and organizing rotation is the topic of the upcoming Configuration and Secret Management series. docker compose config does not print the contents of secret files, but it does print every value that goes into environment through interpolation; before logging config output on CI, make sure no password is left in the environment. We also added .env to .dockerignore: the API's build context is the repo root; today's Dockerfile does not copy .env, but if someone writes COPY . . one day, the passwords end up in a layer of the image.

Common Pitfalls

• Thinking a variable in .env is passed to the container: it only goes where compose.yaml uses it; giving it to the container needs environment or env_file.
• Committing .env: only an empty .env.example goes into the repo; the real values live separately on each machine.
• Not noticing that ${VAR} came in empty: Compose only warns; for required values use ${VAR:?message}.
• Writing the sa password into compose.yaml in plain text: the file goes into the repo and the password belongs to everyone; putting it in an environment variable also carries it into inspect.
• Setting a secret file to 600 and forgetting that the container runs as another user: outside Swarm, uid/mode are not applied, the permission on the host is what counts.
• Trying to shut down services started with a profile using down without the profile: the services and the network stay up.
• Making a default service depend on a service in a profile: while the profile is off, Compose does not start at all and fails with service "api" depends on undefined service "redis": invalid compose project.

A Small Experiment

★☆☆ Delete STORE_TIMEZONE from .env and make sure it is not defined in the shell either. What did Store__TimeZone become in docker compose config? How does the result change when you write - instead of :- and add STORE_TIMEZONE= (empty) to .env?

★★☆ Produce the api-connection-string secret from an environment: BOOKSTORE_CONNECTION_STRING source instead of a file, and define the variable only in the shell. Is the file still under /run/secrets in the container? Can you see the value in docker inspect and in docker compose config?

★★★ Connect Mailpit to Bookstore: add an endpoint to the API that sends a simple "order received" e-mail with MailKit, give the SMTP address from config/api.env (Smtp__Host=mailpit) and see the e-mail at localhost:8025. What should the endpoint do while the tools profile is off?

Configuration now comes in layers, but it all sits in one compose.yaml and the development loop is slow: when one line changes in the API, we rebuild the image and recreate the container. In the next post we decide which part runs in a container and use Compose Watch to carry code changes into the container automatically.