End-to-End Zero Downtime: Deploying Bookstore with Blue-Green and Canary via GitHub Actions

Version note: The examples in this post were run with act 0.2.97, actionlint 1.7.12, .NET 10 (SDK 10.0.403 on the runner), EF (Entity Framework) Core 10.0.14, Nginx 1.30, PostgreSQL 18.6 and Docker 29. The workflow was run locally with act, not on GitHub.

Guess First

Picture a pipeline: build, expand migration, deploy to green, smoke test, 10% canary, 50% canary, full switch. When you run it for the first time, at which step does it fail? Or put differently: how many times does a deploy pipeline that looks right on paper fail on its first runs?

Second question: the pipeline failed at the smoke test, but the migration step had already run. What state is production in right now: a half-finished deploy, or a safe state?

The Reference Repository

In this post we combine all the pieces of the series into a single GitHub Actions workflow. The repository layout:

.
├── .config/dotnet-tools.json
├── .github/workflows/deploy.yml
├── deploy/
│   ├── nginx/nginx.conf
│   ├── canary-gate.sh
│   ├── deploy-idle.sh
│   ├── migration-gate.sh
│   ├── smoke.sh
│   └── traffic.sh
└── src/Bookstore.Api/
    ├── Migrations/
    ├── BookstoreDbContext.cs
    ├── Dockerfile
    ├── Lifecycle.cs
    └── Program.cs

src/Bookstore.Api is the API (Application Programming Interface) from the earlier posts of the series: readiness, drain and graceful shutdown (second post), the version header and BUG_FAIL_RATE for injecting errors (fifth post), EF Core migrations (sixth post). One difference: the version header is now added with Response.OnStarting and UseExceptionHandler is used, so 500 responses carry X-App-Version too. That's required to count the canary metric by version; in the fifth post this header disappeared on failed responses.

The scripts under deploy/ are the pipeline's muscles. Keeping the workflow thin and giving the real work to scripts means the same steps can also be run by hand during an incident. GitHub Actions syntax and features are the topic of the upcoming GitHub Actions series; in particular its posts on environments, deploying to your own server over SSH (Secure Shell) and reusable workflows will cover the production counterpart of what we do here.

The Workflow

name: deploy-production

on:
  workflow_dispatch:
    inputs:
      action:
        description: deploy or rollback
        type: choice
        options: [deploy, rollback]
        default: deploy
      allow_destructive:
        description: approve destructive migrations after review
        type: boolean
        default: false

concurrency:
  group: production
  cancel-in-progress: false

permissions:
  contents: read

jobs:
  build:
    if: inputs.action == 'deploy'
    runs-on: ubuntu-latest
    outputs:
      tag: ${{ steps.meta.outputs.tag }}
    steps:
      - uses: actions/checkout@v7
        with:
          fetch-depth: 0
      - uses: actions/setup-dotnet@v6
        with:
          dotnet-version: 10.0.x
      - id: meta
        run: echo "tag=$(git rev-parse --short HEAD)" >> "$GITHUB_OUTPUT"
      - name: Migration review gate
        env:
          ALLOW_DESTRUCTIVE: ${{ inputs.allow_destructive }}
        run: deploy/migration-gate.sh
      - name: Build the image once
        env:
          TAG: ${{ steps.meta.outputs.tag }}
        run: |
          dotnet publish src/Bookstore.Api -c Release -o src/Bookstore.Api/publish
          docker build -t "depwork-bookstore-api:$TAG" --build-arg APP_VERSION="$TAG" src/Bookstore.Api
      - name: Build the migration bundle
        env:
          RID: linux-${{ runner.arch == 'ARM64' && 'arm64' || 'x64' }}
        run: |
          dotnet tool restore
          dotnet ef migrations bundle --project src/Bookstore.Api --self-contained -r "$RID" -o artifacts/efbundle
      - uses: actions/upload-artifact@v4
        with:
          name: efbundle
          path: artifacts/efbundle

  migrate:
    needs: build
    runs-on: ubuntu-latest
    environment: production
    steps:
      - uses: actions/download-artifact@v4
        with:
          name: efbundle
      - name: Apply expand migrations before the new code
        env:
          DB_CONNECTION: ${{ secrets.DB_CONNECTION }}
        run: |
          chmod +x efbundle
          ./efbundle --connection "$DB_CONNECTION"

  deploy:
    needs: [build, migrate]
    runs-on: ubuntu-latest
    environment: production
    env:
      TAG: ${{ needs.build.outputs.tag }}
      DB_CONNECTION: ${{ secrets.DB_CONNECTION }}
    steps:
      - uses: actions/checkout@v7
      - id: idle
        name: Deploy to the idle color
        run: deploy/deploy-idle.sh "$TAG"
      - name: Smoke test without user traffic
        run: |
          deploy/traffic.sh "${{ steps.idle.outputs.color }}" 0
          deploy/smoke.sh "$TAG"
      - name: Canary 10%
        run: |
          deploy/traffic.sh "${{ steps.idle.outputs.color }}" 10
          deploy/canary-gate.sh "$TAG" 100 120
      - name: Canary 50%
        run: |
          deploy/traffic.sh "${{ steps.idle.outputs.color }}" 50
          deploy/canary-gate.sh "$TAG" 100 120
      - name: Switch all traffic
        run: deploy/traffic.sh "${{ steps.idle.outputs.color }}" 100
      - name: Send traffic back on failure
        if: failure() && steps.idle.outputs.color != ''
        run: deploy/traffic.sh "${{ steps.idle.outputs.color == 'blue' && 'green' || 'blue' }}" 100

  rollback:
    if: inputs.action == 'rollback'
    runs-on: ubuntu-latest
    environment: production
    steps:
      - uses: actions/checkout@v7
      - name: Send all traffic to the previous color
        run: |
          live=$(docker exec depwork-nginx cat /etc/nginx/bookstore/live-color)
          if [ "$live" = blue ]; then previous=green; else previous=blue; fi
          deploy/traffic.sh "$previous" 100

The workflow consists of four jobs, and their order is a summary of this series. build builds the image once, tagged with the commit's short SHA (Secure Hash Algorithm). The same job also produces the migration bundle and uploads it as an artifact, after a migration review gate has run. migrate downloads the bundle and applies it to the database; before the code deploy, because an expand migration doesn't break old code. deploy installs the new version on the idle color, runs a smoke test through the preview port, opens traffic in 10% and 50% steps, and compares canary with control at each step. If any step fails, the last step (if: failure()) gives traffic back to the old color. The rollback job is a manually triggered way back: when action=rollback is chosen with workflow_dispatch, it switches traffic to the previous color.

There are a few more design decisions. concurrency: production prevents two deploys from running at the same time; as we saw in the sixth post, that's the only protection for non-transactional migration steps. permissions: contents: read gives the workflow token the least privilege. Secrets go through env instead of being embedded in run lines with ${{ }}. The bundle's runtime is chosen by the runner's architecture (runner.arch).

On GitHub, environment: production works together with environment protection rules (approvals, wait timers, branch restrictions). act doesn't apply these rules, so I didn't test the approval step here. In a real setup, putting a human approval in front of the migrate job makes sense, especially for releases that contain contract migrations.

Scripts: Traffic, Deploy, Gates

The script that splits traffic in Nginx with weights combines the ideas from the fourth and fifth posts of the series:

#!/usr/bin/env bash
# traffic.sh NEW_COLOR PERCENT: send PERCENT of live traffic to NEW_COLOR, the rest to the other color
set -euo pipefail
new=$1
percent=$2
if [ "$new" = blue ]; then old=green; else old=blue; fi
if [ "$percent" -ge 100 ]; then
    live="    server depwork-$new:8080 resolve;"
    preview=$old
elif [ "$percent" -le 0 ]; then
    live="    server depwork-$old:8080 resolve;"
    preview=$new
else
    live="    server depwork-$old:8080 resolve weight=$((100 - percent));
    server depwork-$new:8080 resolve weight=$percent;"
    preview=$new
fi
docker exec -i depwork-nginx sh -c 'cat > /etc/nginx/bookstore/upstreams.conf' <<CONF
upstream bookstore_live {
    zone bookstore_live 64k;
$live
}

upstream bookstore_preview {
    zone bookstore_preview 64k;
    server depwork-$preview:8080 resolve;
}
CONF
docker exec depwork-nginx nginx -t -q
docker exec depwork-nginx nginx -s reload 2>/dev/null
if [ "$percent" -ge 100 ]; then
    docker exec depwork-nginx sh -c "echo $new > /etc/nginx/bookstore/live-color"
fi
echo "$(date +%T) $new=$percent% $old=$((100 - percent))%"

0% connects the new color only to the preview; 100% gives it all live traffic and moves the old color to the preview. Values in between put both colors live with weights. The Nginx configuration lives in a Docker volume and the script writes it with docker exec -i, so the runner container can talk to Nginx without knowing its file system. On a real server these lines run over SSH.

#!/usr/bin/env bash
# deploy-idle.sh IMAGE_TAG: start the new version on the color that has no traffic
set -euo pipefail
tag=$1
live=$(docker exec depwork-nginx cat /etc/nginx/bookstore/live-color)
if [ "$live" = blue ]; then idle=green; else idle=blue; fi
docker rm -f "depwork-$idle" >/dev/null 2>&1 || true
docker run -d --name "depwork-$idle" --network depwork-net --stop-timeout 40 \
    -e ConnectionStrings__Bookstore="$DB_CONNECTION" \
    -e Shutdown__DrainSeconds=5 \
    "depwork-bookstore-api:$tag" >/dev/null
for _ in $(seq 1 30); do
    if docker exec depwork-nginx wget -q -O /dev/null "http://depwork-$idle:8080/health/ready" 2>/dev/null; then
        echo "color=$idle" >> "${GITHUB_OUTPUT:-/dev/null}"
        echo "depwork-$idle is ready with $tag"
        exit 0
    fi
    sleep 1
done
echo "depwork-$idle did not become ready" >&2
exit 1

#!/usr/bin/env bash
# canary-gate.sh NEW_VERSION MIN_REQUESTS MAX_SECONDS: wait for enough canary traffic, then compare 5xx rates
set -euo pipefail
new=$1
min=$2
end=$((SECONDS + $3))
docker exec depwork-nginx sh -c ': > /var/log/nginx/bookstore.log'
while true; do
    sleep 5
    read -r cn ce sn se < <(docker exec depwork-nginx cat /var/log/nginx/bookstore.log | awk -v new="$new" '
        $3 == new { cn++; if ($2 >= 500) ce++; next }
        { sn++; if ($2 >= 500) se++ }
        END { print cn + 0, ce + 0, sn + 0, se + 0 }')
    if [ "$cn" -ge "$min" ]; then
        break
    fi
    if [ $SECONDS -ge $end ]; then
        echo "only $cn canary requests in $3 s, not enough to decide"
        exit 1
    fi
done
awk -v cn="$cn" -v ce="$ce" -v sn="$sn" -v se="$se" 'BEGIN {
    c = 100 * ce / cn; s = sn ? 100 * se / sn : 0
    printf "new: %d requests, %.2f%% 5xx | old: %d requests, %.2f%% 5xx\n", cn, c, sn, s
    if (c - s > 1.0) { print "canary is worse than baseline"; exit 1 }
    print "canary looks healthy"
}'

#!/usr/bin/env bash
# migration-gate.sh: fail when migrations added since the last release drop or rename something
set -euo pipefail
last=$(git describe --tags --abbrev=0 --match 'release-*' 2>/dev/null || git rev-list --max-parents=0 HEAD)
files=$(git diff --name-only --diff-filter=A "$last" HEAD -- 'src/*/Migrations/*.cs' | grep -v '\.Designer\.cs$' || true)
echo "migrations since $last: ${files:-none}"
[ -z "$files" ] && exit 0
# only Up() matters; Down() of an additive migration always contains Drop*
if awk '/override void Up\(/,/override void Down\(/' $files | grep -nE 'Drop(Column|Table|Index)|Rename(Column|Table)'; then
    if [ "${ALLOW_DESTRUCTIVE:-false}" != "true" ]; then
        echo "::error::Destructive migration found; run again with allow_destructive after review"
        exit 1
    fi
    echo "Destructive migration approved by the run input."
fi

Running It Locally with act

I ran the pipeline locally with act without pushing it to GitHub. The environment: on the depwork-net Docker network, PostgreSQL, Nginx and a depwork-blue container running the first version, plus a load container sending about 18 requests per second. It imitates real users, because the canary gate needs traffic to decide. The command:

act workflow_dispatch -W .github/workflows/deploy.yml --input action=deploy \
    -P ubuntu-latest=ghcr.io/catthehacker/ubuntu:act-latest \
    --network depwork-net \
    --container-daemon-socket unix:///var/run/docker.sock \
    --secret-file act.secrets \
    --artifact-server-path act-artifacts \
    --pull=false

--network attaches the job containers to the same Docker network, and --container-daemon-socket passes the socket in so the jobs can use Docker. Because this machine uses Docker Desktop, I also told act where the socket is with DOCKER_HOST. In production, instead of the runner accessing the Docker socket, the image is pushed to a registry (for example GHCR (GitHub Container Registry)) and the server pulls it from there. Here build and deploy share the same Docker daemon, so the "build once" principle holds without a registry: the deploy job uses the image the build job produced.

The First Runs: Why Did the Pipeline Fail Four Times?

The answer to the first opening question: this pipeline failed four times before its successful run, and each failure repeated a lesson from this series.

The first failure was in the migration gate:

[deploy-production/build   ]   | migrations since release-1: src/Bookstore.Api/Migrations/20261108041219_AddCustomerNameParts.cs
[deploy-production/build   ]   | 29:            migrationBuilder.DropColumn(
[deploy-production/build   ]   | 33:            migrationBuilder.DropColumn(
[deploy-production/build   ]   ❗  ::error::Destructive migration found; run again with allow_destructive after review
[deploy-production/build   ]   ❌  Failure - Main Migration review gate [77.9335ms]

AddCustomerNameParts only adds columns, but the first version of the gate scanned the whole file. The Down() method of every migration that adds something contains DropColumn. The fix was to scan only the Up() block (the awk line above). A gate's false alarms get it switched off quickly, so the gate needs to be tested with real migrations.

The second failure was in the artifact upload:

[deploy-production/build   ] ⭐ Run Main actions/upload-artifact@v7
time="2026-11-08T07:15:11+03:00" level=error msg="Error decode request body: proto: (line 1:96): unknown field \"mime_type\""
[deploy-production/build   ]   | Attempt 2 of 5 failed with error: Unexpected end of JSON input. Retrying request in 16970 ms...

act 0.2.97's local artifact server doesn't recognize the mime_type field sent by actions/upload-artifact@v7, and the step fails after five attempts. I pinned the artifact actions to v4 in the workflow. On GitHub's own infrastructure v7 works fine; this is a concrete example of the difference between act and GitHub.

The third failure was the most instructive:

[deploy-production/migrate ]   | Applying migration '20261108041219_AddCustomerNameParts'.
[deploy-production/deploy  ]   | depwork-green is ready with f266e33
[deploy-production/deploy  ]   | /health/ready -> 200
[deploy-production/deploy  ]   | /api/books -> 200
[deploy-production/deploy  ]   | /api/customers/1 -> 200
[deploy-production/deploy  ]   | version -> c5a0dfb
[deploy-production/deploy  ]   ❌  Failure - Main Smoke test without user traffic [278.967084ms]
[deploy-production/deploy  ] ⭐ Run Main Send traffic back on failure
[deploy-production/deploy  ]   | 04:17:51 blue=100% green=0%

The smoke test passed the health checks but failed the version check: the preview port still pointed at the old blue. The script had brought the new color up but forgot to point Nginx's preview upstream at it. In the fourth post we said "don't skip the version check"; this is exactly why. The fix was calling traffic.sh <color> 0 before the smoke test. The answer to the second opening question is in this output too: the migration (AddCustomerNameParts) had been applied and the code deploy failed. But production was safe, because after the expand migration the old code on blue kept working without problems. Only the deploy was half-finished, not the system. That's exactly the value of expand-contract.

The fourth failure was in the canary gate:

[deploy-production/deploy  ] ⭐ Run Main Canary 10%
[deploy-production/deploy  ]   | 04:19:34 green=10% blue=90%
[deploy-production/deploy  ]   | new: 36 requests, 0.00% 5xx | old: 324 requests, 0.00% 5xx
[deploy-production/deploy  ]   | not enough canary traffic
[deploy-production/deploy  ]   ❌  Failure - Main Canary 10% [20.362683833s]

The gate waited for a fixed twenty-second window, and the 10% canary saw only 36 requests. The statistical trap from the fifth post became real here: you don't decide on a small sample. I changed the gate to "wait until at least 100 canary requests arrive, at most 120 seconds". With little traffic the pipeline slows down, but it doesn't decide wrongly.

The Successful Run

A summary of the run after the four fixes:

[deploy-production/build   ]   ✅  Success - Main Migration review gate [69.789667ms]
[deploy-production/build   ]   ✅  Success - Main Build the image once [14.92709125s]
[deploy-production/build   ]   ✅  Success - Main Build the migration bundle [14.985959459s]
[deploy-production/build   ]   ✅  Success - Main actions/upload-artifact@v4 [4.200684875s]
[deploy-production/migrate ]   | No migrations were applied. The database is already up to date.
[deploy-production/migrate ]   ✅  Success - Main Apply expand migrations before the new code [548.045625ms]
[deploy-production/deploy  ]   | depwork-green is ready with 7b2c3f6
[deploy-production/deploy  ]   | 04:21:19 green=0% blue=100%
[deploy-production/deploy  ]   | /health/ready -> 200
[deploy-production/deploy  ]   | /api/books -> 200
[deploy-production/deploy  ]   | /api/customers/1 -> 200
[deploy-production/deploy  ]   | version -> 7b2c3f6
[deploy-production/deploy  ]   | 04:21:20 green=10% blue=90%
[deploy-production/deploy  ]   | new: 103 requests, 0.00% 5xx | old: 922 requests, 0.00% 5xx
[deploy-production/deploy  ]   | canary looks healthy
[deploy-production/deploy  ]   | 04:22:16 green=50% blue=50%
[deploy-production/deploy  ]   | new: 138 requests, 0.00% 5xx | old: 139 requests, 0.00% 5xx
[deploy-production/deploy  ]   | canary looks healthy
[deploy-production/deploy  ]   | 04:22:36 green=100% blue=0%
[deploy-production/deploy  ] 🏁  Job succeeded

The migration step said "already up to date", because the expand migration had been applied on the third run. A bundle being safely re-runnable pays off here. The 10% step waited about a minute to reach 103 canary requests; at the 50% step both colors took almost equal traffic. After the pipeline finished:

curl -s localhost:18600/api/version
curl -s localhost:18600/api/customers/1

{"version":"7b2c3f6","instance":"269aa4811c9d"}
{"id":1,"firstName":"Alan","lastName":"Lovelace","email":"user1@example.com"}

The new version is live and reads customer data from the new columns.

The Manual Rollback Job

Imagine noticing a problem after the pipeline finished. The way back is the same workflow's rollback action:

act workflow_dispatch -W .github/workflows/deploy.yml --input action=rollback \
    -P ubuntu-latest=ghcr.io/catthehacker/ubuntu:act-latest \
    --network depwork-net \
    --container-daemon-socket unix:///var/run/docker.sock \
    --secret-file act.secrets \
    --artifact-server-path act-artifacts \
    --pull=false

[deploy-production/rollback] ⭐ Run Main Send all traffic to the previous color
[deploy-production/rollback]   | 04:24:18 blue=100% green=0%
[deploy-production/rollback]   ✅  Success - Main Send all traffic to the previous color [3.50226225s]
[deploy-production/rollback] 🏁  Job succeeded
{"version":"c5a0dfb","instance":"a132bfe009c8"}
{"id":1,"fullName":"Alan Lovelace","email":"user1@example.com"}

Traffic went back to the old color in 3.5 seconds, and the old code returns the customer with fullName on the expanded schema. Rollback can be this boring because the contract migration isn't part of this release.

I also tried what happens when we put the contract into the next release. I added the migration that drops FullName on a separate branch and ran the workflow:

[deploy-production/build   ]   | migrations since release-2: src/Bookstore.Api/Migrations/20261108042437_ContractCustomerFullName.cs
[deploy-production/build   ]   | 3:            migrationBuilder.DropColumn(
[deploy-production/build   ]   ❗  ::error::Destructive migration found; run again with allow_destructive after review
[deploy-production/build   ]   ❌  Failure - Main Migration review gate [130.3645ms]
[deploy-production/build   ] 🏁  Job failed

The gate stopped the pipeline at the build step. This release only moves forward once a human reviews the migration and restarts it with the allow_destructive input. That review asks the preconditions from the sixth post: are all instances on a version that doesn't use FullName? Is the backfill complete? Is a way back like the rollback we just did still possible? After the rollback, blue still reads FullName, so approving the contract in this state would break production.

In Production

Three things change when you take this pipeline to a real environment. The image is pushed to a registry and the deploy step runs on the server (over SSH or through an agent). The smoke test and canary gate feed on a real metrics system (Prometheus, Azure Monitor) instead of the Nginx log; that's a topic for the upcoming Observability series. And approval steps move to GitHub environments. For teams using Jenkins the mapping is direct: each job becomes a stage, the concurrency group corresponds to the lock step, and if: failure() maps to a post { failure { ... } } block. The scripts stay as they are; that's one more benefit of leaning the pipeline on scripts.

Measure the pipeline itself as well: the duration of each run, which step it failed at, and how many manual rollbacks happened. That's the raw data for the DORA (DevOps Research and Assessment) metrics in the eighth post of the series.

Common Pitfalls

Forgotten cache invalidation. If the new version changed the format of a cache key, values written by the old version are still there. You need a version prefix on the key (v2:book:42) or to clear the relevant keys during the deploy. Otherwise a bug that never showed up in the canary explodes at 100%.

Old Angular chunks. Switching the API with blue-green doesn't save the frontend's old tabs. Keeping hashed files for a while and reloading the page on a chunk error, as in the second post of the series, must be part of the pipeline's frontend publishing step.

Migration order. Putting a contract migration in the same release as the code makes rollback impossible. In this pipeline the gate stops it, but the real protection is in release planning: expand in this release, contract in the next.

Forgetting the pipeline can stop halfway. Every step must be safely re-runnable (idempotent). The bundle can say "already up to date", traffic.sh can write the same percentage again, and deploy-idle.sh removes the old idle container and starts it again. That's what makes restarting a failed pipeline from the beginning safe.

A Small Experiment

★☆☆ Add another workflow_dispatch input to the workflow: canary_steps (for example 10,50). Write a loop that reads the steps from that input and run it with 5,25,50.

★★☆ Prepare a version with BUG_FAIL_RATE=0.05 (add it as an ENV in the Dockerfile), run the workflow, and watch the gate fail at the 10% step and the Send traffic back on failure step give traffic back to the old color.

★★★ Design a second workflow that ships the contract release safely: first a job that checks "are all instances on the new version?" and "is the empty-row count 0?", then the approved contract migration. Tie the gate to these checks.

The series ends here. We went from recreate's measured two-and-a-half-second outage to a pipeline that rolls a bug back in ten seconds and leaves the contract migration to a human's approval. Tools change; a cloud load balancer replaces Nginx, GitHub's own runners replace act. But the order of the questions stays the same: is the app ready, can two versions live at the same time, is the schema backward compatible, how many users does a bug touch and for how long, and is the way back really open?