Multi-stage Builds in Depth: Targets, a Test Stage, Parallel Stages and Image Size

Version note: The examples in this post were run with Docker Engine 29.9.3 (BuildKit 0.34.2), Buildx 0.39.0, the .NET SDK 10.0.404 and .NET Runtime 10.0.15 images, xunit.v3 4.0.3 (Microsoft.Testing.Platform) and registry:3. Sizes were taken from linux/amd64 images.

Guess First

A Dockerfile has five stages: restore, build, test, runtime and final. The test stage runs dotnet test, while final derives from runtime and takes no files from test. When you run docker build ., do the tests run? Second question: the Bookstore.Tools image was 1.34 GB at the end of post 6. Once the SDK leaves the final image, how many MB does it shrink to: 600, 300 or 150?

Why Split into Stages?

The image of Bookstore.Tools, the import tool of Bookstore, the online bookstore used as the shared example across this site's series, currently carries everything: the .NET SDK, NuGet-related intermediates, the source code, the obj/ and bin/ folders. The only thing running in the container is dotnet Bookstore.Tools.dll, and the runtime plus the publish output are enough for that. The extra is not just size: anyone who pulls the image can read the source code, the CVEs of every package in the SDK end up in the scan report, and tools an attacker could use (a compiler, a shell, curl) are ready and waiting.

The idea of a multi-stage build is simple: you write several FROM lines in one Dockerfile, each starts a new stage, and the final image is made only from the content of the last stage. The only way to take files from earlier stages is COPY --from=<stage>. In the ASP.NET Core Web API series post "Containerizing an ASP.NET Core Web API with Docker and Setting Up CI/CD with GitHub Actions" we saw a basic two-stage example; here we extend the same idea on Bookstore: a separate test stage, compiling for the target architecture, a stage that hands output to the host, and a separate target for debugging.

The Stages for Bookstore.Tools

First we added tests: tests/Bookstore.Tools.Tests tests BookRow's ISBN-13 validation and the Trim() fix we added in post 6. The whole Dockerfile:

# syntax=docker/dockerfile:1
ARG DOTNET_VERSION=10.0

# restore: SDK runs on the build machine's platform, never under QEMU
FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:${DOTNET_VERSION} AS restore
WORKDIR /src
COPY Bookstore.slnx global.json ./
COPY src/Bookstore.Tools/Bookstore.Tools.csproj src/Bookstore.Tools/
COPY tests/Bookstore.Tools.Tests/Bookstore.Tools.Tests.csproj tests/Bookstore.Tools.Tests/
RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages \
    dotnet restore Bookstore.slnx
COPY src/ src/

# build: publish for the target architecture (cross-compilation)
FROM restore AS build
ARG TARGETARCH
ARG BUILD_CONFIGURATION=Release
ARG APP_VERSION=1.0.0
RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages \
    dotnet publish src/Bookstore.Tools/Bookstore.Tools.csproj -c $BUILD_CONFIGURATION -a $TARGETARCH \
        -o /app -p:Version=$APP_VERSION

# test: only built when asked for (--target test or the bake "test" target)
FROM restore AS test
COPY tests/ tests/
RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages \
    dotnet test --project tests/Bookstore.Tools.Tests/Bookstore.Tools.Tests.csproj --no-restore

# export: publish output to the host (docker build --target export --output out)
FROM scratch AS export
COPY --from=build /app /

# runtime: runtime only, no SDK, no shell, non-root
FROM mcr.microsoft.com/dotnet/runtime:${DOTNET_VERSION}-noble-chiseled AS runtime
ARG APP_VERSION=1.0.0
WORKDIR /app
COPY --from=build /app ./
COPY --chown=app:app data/books.csv data/books.csv
LABEL org.opencontainers.image.title="Bookstore.Tools" \
      org.opencontainers.image.version=$APP_VERSION
ENV DOTNET_ENVIRONMENT=Production \
    Import__File=data/books.csv
USER app
ENTRYPOINT ["dotnet", "Bookstore.Tools.dll"]
CMD ["--Import:DelayMs=500"]

# debug: runtime plus a static busybox taken from another image
FROM runtime AS debug
COPY --from=busybox:1.37-musl /bin/busybox /busybox/busybox

# final: the last stage is the default target, so it must be the production image
FROM runtime AS final

A few details:

• ARG DOTNET_VERSION is declared before the first FROM; such an ARG can only be used in FROM lines and lets you change the .NET version in one place. To use it inside a stage you have to declare ARG there again; that is why we declare APP_VERSION in two stages separately.
• Names such as AS restore and AS build are given to stages, and FROM restore AS build builds one stage on top of another. build and test both derive from the same restore stage, so restore happens once.
• global.json is in the first COPY because it decides which test platform dotnet test runs with (more on that shortly). The rule from post 6 holds: files that affect restore and SDK behavior come before the source code.
• The runtime image is the chiseled variant based on Ubuntu 24.04: it has only the packages the .NET runtime needs, no shell and no package manager, and an app user is defined. We will compare the image families in detail in post 8.

Which Stages Run? Targets and Parallelism

BuildKit does not run a Dockerfile line by line from top to bottom; it works backwards from the target stage to a dependency graph and builds only the stages that lead to that target. Without a target, the last stage is the target. That is why the answer to the first question is no:

docker build --progress=plain -f src/Bookstore.Tools/Dockerfile -t dockerseries-bookstore-tools:1.0.0 .

#8 [restore 1/7] FROM mcr.microsoft.com/dotnet/sdk:10.0@sha256:fe0fd4e93bd2fbb47c883b7a6369e5c8d3db8bf01cdc5981f6ee35fbdf3ff4fb
#9 [runtime 1/4] FROM mcr.microsoft.com/dotnet/runtime:10.0-noble-chiseled@sha256:1b234cef150624fe8cc7d027e8d742c0f03a16aef2484dc469eb8c872167255e
#14 [restore 6/7] RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages     dotnet restore Bookstore.slnx
#14 CACHED
#15 [restore 7/7] COPY src/ src/
#16 [build 1/1] RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages     dotnet publish src/Bookstore.Tools/Bookstore.Tools.csproj -c Release -a amd64         -o /app -p:Version=1.0.0
#16 0.629   Restored /src/src/Bookstore.Tools/Bookstore.Tools.csproj (in 125 ms).
#16 DONE 1.3s
#17 [runtime 2/4] WORKDIR /app
#18 [runtime 3/4] COPY --from=build /app ./
#19 [runtime 4/4] COPY --chown=app:app data/books.csv data/books.csv
#20 naming to docker.io/library/dockerseries-bookstore-tools:1.0.0 done

There is no test, export or debug in the output; BuildKit never ran them. Since the final stage has no steps of its own, the log shows the steps under the name runtime. Because TARGETARCH comes from the target platform, publish ran with -a amd64. The short restore inside publish (125 ms) is the extra restore the SDK does for the target architecture when -a is given; it is fast because the packages come from the cache mount. There is also parallelism: stages that do not depend on each other run at the same time. It does not show in this output because restore came from the cache; in the first build, pulling the runtime image started together with restore and finished before it. The runtime stage waits for build only at the COPY --from=build line.

To run the tests, you name the target explicitly:

docker build --progress=plain --target test -f src/Bookstore.Tools/Dockerfile .

#14 [test 1/2] COPY tests/ tests/
#15 [test 2/2] RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages     dotnet test --project tests/Bookstore.Tools.Tests/Bookstore.Tools.Tests.csproj --no-restore
#15 2.478 Test run summary: Passed!
#15 2.478   total: 6
#15 2.478   failed: 0
#15 2.478   succeeded: 6
#15 2.478   skipped: 0
#15 2.478   duration: 646ms

The restore stage came CACHED from the previous build; only the tests ran. If tests fail, dotnet test exits with a non-zero code and the build stops. We added a test target to the Bake file from post 6 and put it into the ci group:

target "test" {
  context    = "."
  dockerfile = "src/Bookstore.Tools/Dockerfile"
  target     = "test"
  output     = ["type=cacheonly"]
}

type=cacheonly does not store the result as an image, it only writes to the build cache; we do not want an image from the test stage, only for it to succeed. We also added target = "final" to the tools target, so it stays clear which image is produced even when new stages are added to the Dockerfile. Running the group, test and build move side by side in the same session:

docker buildx bake ci

#13 [test restore 6/7] RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages     dotnet restore Bookstore.slnx
#22 [test test 1/2] COPY tests/ tests/
#23 [tools runtime 1/4] FROM mcr.microsoft.com/dotnet/runtime:10.0-noble-chiseled@sha256:1b234cef150624fe8cc7d027e8d742c0f03a16aef2484dc469eb8c872167255e
#24 [tools build 1/1] RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages     dotnet publish ...
#25 [test test 2/2] RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages     dotnet test --project tests/Bookstore.Tools.Tests/Bookstore.Tools.Tests.csproj --no-restore
#25 2.336 Test run summary: Passed!
#28 [tools runtime 4/4] COPY --chown=app:app data/books.csv data/books.csv
#29 naming to docker.io/library/dockerseries-bookstore-tools:1.0.0 done
lint
Check complete, no warnings found.

Publish (#24) and test (#25) started at the same time; the shared restore stage ran once. Note that the tools target does not wait for the test result. To try it, we deliberately changed digit * 3 in HasValidIsbn13 to digit * 2 and ran docker buildx bake ci again:

#29 naming to docker.io/library/dockerseries-bookstore-tools:1.0.0 done
#24 2.125 Test run summary: Failed!
#24 2.125   failed: 2
ERROR: target test: failed to solve: process "/bin/sh -c dotnet test --project tests/Bookstore.Tools.Tests/Bookstore.Tools.Tests.csproj --no-restore" did not complete successfully: exit code: 2

Bake failed, but by then the dockerseries-bookstore-tools:1.0.0 tag had already moved to the new image built with the broken code (its image ID changed in docker image ls). If you want a "no image unless tests pass" rule, you build it with the step order of the pipeline: the test target first, the image only if it succeeds. Tying the final image artificially to the test stage (for example copying an empty file from it) also works, but you lose the parallelism.

The Test Stage, and Why a Separate CI Job Is Still Needed

The test project uses xunit.v3. Version 4.x of xunit.v3 runs on Microsoft.Testing.Platform, and on the .NET 10 SDK dotnet test needs a global.json at the repository root to run in this mode. We forgot it on the first try, and the test stage stopped with this error:

error : Testing with VSTest target is no longer supported by Microsoft.Testing.Platform on .NET 10 SDK and later. If you use dotnet test, you should opt-in to the new dotnet test experience. For more information, see https://aka.ms/dotnet-test-mtp-error

{
  "test": {
    "runner": "Microsoft.Testing.Platform"
  }
}

In this mode dotnet test takes the project path through --project. The tests themselves are ordinary xUnit tests:

using Bookstore.Tools;
using Xunit;

namespace Bookstore.Tools.Tests;

public sealed class BookRowTests
{
    [Theory]
    [InlineData("9780441172719", true)]
    [InlineData("9780141439587", true)]
    [InlineData("9780141439510", false)]
    [InlineData("978014143958", false)]
    [InlineData("97801414395AB", false)]
    public void HasValidIsbn13_checks_length_digits_and_check_digit(string isbn, bool expected)
    {
        BookRow row = new(isbn, "Any", "Any", 100m);

        Assert.Equal(expected, row.HasValidIsbn13());
    }

    [Fact]
    public void Parse_trims_fields_and_reads_invariant_price()
    {
        BookRow row = BookRow.Parse("9780441172719, Dune ,Frank Herbert,420.50");

        Assert.Equal("Dune", row.Title);
        Assert.Equal(420.50m, row.PriceTry);
    }
}

The test stage has two benefits: tests run on the same SDK version and the same restore result as the production image, and test dependencies (xunit, the test project, the bin/Debug folder) never reach the final image. Even so, we prefer to run tests in a separate CI job (dotnet test directly on the runner). The reasons are practical: test result files and the coverage report stay inside the build container and need an extra --output setting to get out; the output of a failing test shows up truncated in the BuildKit log; integration tests that need a database cannot run inside docker build, because build steps have no permission to start containers (that is the Testcontainers topic of the Integration and E2E Test series). The test stage is most useful for closing the question "is the code this Dockerfile builds really the code that was tested?" and for running everything with one command during local development.

COPY --from, FROM scratch and the Distroless Idea

COPY --from does not only take a stage name; it can also take an image reference. In the debug stage we took a single file, the statically compiled busybox binary, from the busybox:1.37-musl image. The chiseled image shows why we need it:

docker run --rm --entrypoint sh dockerseries-bookstore-tools:1.0.0
docker build --target debug -f src/Bookstore.Tools/Dockerfile -t dockerseries-bookstore-tools:1.0.0-debug .
docker run --rm --entrypoint /busybox/busybox dockerseries-bookstore-tools:1.0.0-debug ls -ln /app/data

docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: exec: "sh": executable file not found in $PATH
total 4
-rw-r--r--    1 1654     1654           314 Nov 14 11:01 books.csv

The production image has no shell; you cannot get in with docker exec and look around. That is intentional: in an image without a shell, a process an attacker takes over has no tools to use either. When you need to look, you build a separate debug target; the image that goes to production does not change. Ways to debug a shell-less image without rebuilding it (docker debug, a helper container attached to the same namespaces) come in post 14.

Carrying tools this way is common: a static healthcheck binary, a certificate bundle or a CLI tool from another image. The rule is that the copied file must be able to run in the target image: a dynamically linked binary that needs a libc the target does not have will not run. That is why we picked the musl variant of busybox.

FROM scratch means an empty image: there are no files in it. The export stage uses it to take a build output to the host. With --output, BuildKit writes the target stage's file system to a folder instead of an image:

docker build --target export --output type=local,dest=out -f src/Bookstore.Tools/Dockerfile .
ls out | head -5

#16 exporting to client directory
#16 copying files 1.34MB 0.0s done
Bookstore.Tools
Bookstore.Tools.deps.json
Bookstore.Tools.dll
Bookstore.Tools.pdb
Bookstore.Tools.runtimeconfig.json

This is a short way to compile and get the output in an environment without the .NET SDK (for example a build agent that only has Docker). The out/ folder has been in .dockerignore since post 5, so it does not enter the next build's context. scratch is used as a final image too, but only for static binaries that depend on nothing (such as Go programs). Because the .NET runtime needs a libc and a few system libraries, the .NET counterpart of the idea is distroless images: images without a package manager or shell, carrying only the files the application needs to run. Microsoft's Ubuntu chiseled and Azure Linux distroless images are in this class.

Image Size: What Did We Gain?

The answer to the second question. All images were measured for linux/amd64 with docker image ls --tree:

The answer is the one closest to 150: from 1.34 GB to 143 MB, about one ninth. The compressed size pulled from the registry (CONTENT SIZE) went from 367 MB to 44.1 MB; that is the data pulled on every deploy and every new node. In the containerd image store, DISK USAGE is the sum of the compressed content and the unpacked layers; for our image, 143 MB is roughly 99 MB of unpacked files plus 44 MB of compressed content. One more thing stands out in the table: our application adds only 1.4 MB to the base image. Image size is now decided almost entirely by the choice of base image. runtime-deps does not even contain the .NET runtime; it is for self-contained or Native AOT applications.

To see what came from where, layer by layer, docker history is enough, as in post 5:

docker history dockerseries-bookstore-tools:1.0.0

IMAGE          CREATED              CREATED BY                                      SIZE      COMMENT
b3faf6b94f0d   About a minute ago   CMD ["--Import:DelayMs=500"]                    0B        buildkit.dockerfile.v0
<missing>      About a minute ago   ENTRYPOINT ["dotnet" "Bookstore.Tools.dll"]     0B        buildkit.dockerfile.v0
<missing>      About a minute ago   USER app                                        0B        buildkit.dockerfile.v0
<missing>      About a minute ago   ENV DOTNET_ENVIRONMENT=Production Import__Fi…   0B        buildkit.dockerfile.v0
<missing>      About a minute ago   LABEL org.opencontainers.image.title=Booksto…   0B        buildkit.dockerfile.v0
<missing>      About a minute ago   COPY --chown=app:app data/books.csv data/boo…   16.4kB    buildkit.dockerfile.v0
<missing>      About a minute ago   COPY /app ./ # buildkit                         1.41MB    buildkit.dockerfile.v0
<missing>      About a minute ago   WORKDIR /app                                    8.19kB    buildkit.dockerfile.v0
<missing>      About a minute ago   ARG APP_VERSION=1.0.0                           0B        buildkit.dockerfile.v0
<missing>      3 days ago           CMD ["--info"]                                  0B        buildkit.dockerfile.v0
<missing>      3 days ago           ENTRYPOINT ["/usr/bin/dotnet"]                  0B        buildkit.dockerfile.v0
<missing>      3 days ago           COPY /dotnet-symlink /usr/bin # buildkit        12.3kB    buildkit.dockerfile.v0
<missing>      3 days ago           COPY /usr/share/dotnet /usr/share/dotnet # b…   83.2MB    buildkit.dockerfile.v0
<missing>      3 days ago           ENV DOTNET_VERSION=10.0.15                      0B        buildkit.dockerfile.v0
<missing>      3 days ago           USER 1654                                       0B        buildkit.dockerfile.v0
<missing>      3 days ago           COPY /rootfs/home/app /home/app # buildkit      12.3kB    buildkit.dockerfile.v0
<missing>      3 days ago           ENV APP_UID=1654 ASPNETCORE_HTTP_PORTS=8080 …   0B        buildkit.dockerfile.v0
<missing>      3 days ago           COPY /rootfs / # buildkit                       14.4MB    buildkit.dockerfile.v0

The base image itself is built multi-stage: the COPY /rootfs / and COPY /usr/share/dotnet lines are content copied from other stages in Microsoft's Dockerfile. A 14.4 MB root file system and the 83.2 MB .NET runtime, with our 1.41 MB publish output on top. If you want to see which file inside a layer takes how much space, tools such as dive show the layers as a file tree; but docker history answers most questions.

Multi-Architecture Builds Without QEMU

In post 6 we said we should not run the .NET SDK under QEMU. In this Dockerfile three lines take care of that. FROM --platform=$BUILDPLATFORM always runs the SDK stages on the build machine's architecture. ARG TARGETARCH receives the architecture name (amd64, arm64) BuildKit fills in for each target platform, and dotnet publish -a $TARGETARCH produces output for that architecture. The FROM of the final stage has no --platform, so that stage uses the runtime image of the target platform. No emulation is needed, because the .NET SDK can compile for another architecture by itself. We added two platforms to the tools-release target in the Bake file:

target "tools-release" {
  inherits   = ["tools"]
  platforms  = ["linux/amd64", "linux/arm64"]
  tags       = ["${REGISTRY}/bookstore/tools:${TAG}"]
  cache-from = ["type=registry,ref=${REGISTRY}/bookstore/tools:buildcache"]
  cache-to   = ["type=registry,ref=${REGISTRY}/bookstore/tools:buildcache,mode=max"]
  output     = ["type=registry"]
}

docker buildx bake tools-release
docker buildx imagetools inspect localhost:5000/bookstore/tools:1.0.0
docker run --rm --platform linux/arm64 localhost:5000/bookstore/tools:1.0.0

Name:      localhost:5000/bookstore/tools:1.0.0
  Name:        localhost:5000/bookstore/tools:1.0.0@sha256:b364362899ccf588a2638eb9ffaf9122923849efc85e7827bc08e4c65a15db73
  Platform:    linux/amd64
  Name:        localhost:5000/bookstore/tools:1.0.0@sha256:efccdbf15b9c0646577649efe26e5a985dff50f12df34b3c2181e2479d75a146
  Platform:    linux/arm64
...
11:22:24 info: Bookstore.Tools.ImportWorker[0] Import finished: 5 books in 3 batches

In the build log the restore stage runs once; the publish step appears on a separate line per target, in the form [<build platform>-><target> build 1/1], one running with -a amd64 and the other with -a arm64. When you run the arm64 image on an x64 machine, the runtime itself runs under QEMU; that is acceptable for this check, but the real place to test the arm64 image is an arm64 machine. In post 9 we will do the same without a Dockerfile, using the SDK's own container support.

The approach has a limit: publishing with -a picks the target architecture's native file when the app uses a NuGet package with a native library (for example SQLite or an image processing library), which is the right behavior. But if the build stage has to run something for the target architecture (for example calling a target-specific tool with RUN), that step falls back to QEMU again. In .NET this is rare; in the Angular build it never comes up, because the output is architecture-independent JavaScript.

In Production

Keep the final stage, and which stage produces which image, explicit. In Bake or the pipeline always pass --target final (or target = "final"), so the image going to production does not change silently when someone adds a new stage at the bottom of the Dockerfile.

Check that no SDK, test project, source code or build leftover other than .pdb files reaches the production image. An unexpected large layer in docker history, or unexpected files listed with docker run --entrypoint, are the signs. Many teams keep .pdb files on purpose because they give line numbers in stack traces; if you do not want them, add -p:DebugType=none to publish.

Pick the base image by what the application really needs. Chiseled images have no shell, no curl, no ICU and no time zone data. That is fine for Bookstore.Tools; for Bookstore.Api, which sorts in Turkish and uses the Europe/Istanbul time zone, we will see in the next post how that turns into a bug.

Push the debug target to the registry under a separate tag (for example 1.0.0-debug), or do not push it at all. If they share a tag, you cannot tell which one runs in production.

Common Pitfalls

Deriving the final stage from the SDK stage. Writing FROM build AS final throws away everything multi-stage gives you and the image is back above 1 GB. The FROM of the final stage must always be a runtime image.

Referring to stages by index instead of name. COPY --from=0 points to the first stage; when a new stage is added at the top of the Dockerfile, the same line silently starts copying from another stage. Give every stage a name with AS and refer to it by name.

Writing the debug or test stage at the bottom. A docker build without a target builds the last stage; if debug is at the bottom, the image with busybox goes to production. That is why the last stage of our Dockerfile is final.

Declaring an ARG before FROM and trying to use it inside a stage. An ARG before the first FROM is only valid in FROM lines. In a small test (ARG DOTNET_VERSION=10.0, then FROM alpine:3 and RUN echo "version=$DOTNET_VERSION"), docker build --check said "no warnings found" while the build printed version=: the value was empty. If you need it inside a stage, declare ARG DOTNET_VERSION there again without a value; it takes the default from above.

Assuming the test stage runs on every build. As we saw above, BuildKit builds only the stages that lead to the target. To make sure tests really run in CI, call them as a separate step or Bake target.

A Small Experiment

★☆☆ Change digit * 3 in BookRow.HasValidIsbn13 to digit * 2. Run docker build . first, then docker build --target test .. Which one caught the bug, and why?

★★☆ Set the final stage's base image to runtime:10.0, runtime:10.0-alpine and runtime:10.0-noble-chiseled in turn; measure the linux/amd64 size of each with docker image ls --tree and check that the app runs on all three.

★★★ In the build stage, publish with -p:PublishSingleFile=true --self-contained and base the final image on runtime-deps:10.0-noble-chiseled. How did you have to change ENTRYPOINT, and how much did the image shrink or grow?

Bookstore.Tools is now a 143 MB image with no shell, running as non-root, built for two architectures without QEMU. In the next post we apply the same approach to the real application, Bookstore.Api: ASP.NET Core image families, port 8080, configuration through environment variables, and why Turkish sorting and the time zone break in a chiseled image.