Mert Özen Further With Every Line
DevOps Kubernetes — #17

End to End: Taking a .NET + Angular App to Managed Kubernetes (AKS, EKS, GKE) and a Production Checklist

Mert Özen Nov 8, 2026 11 min 5 views
End to End: Taking a .NET + Angular App to Managed Kubernetes (AKS, EKS, GKE) and a Production Checklist

On managed Kubernetes the control plane stays with the provider; if you use node identity for image pulls and workload identity for cloud access, not a single cloud secret is left in the pipeline or the cluster. We cover AKS, EKS and GKE differences, an OIDC deploy-to-AKS workflow validated with actionlint, deprecated API clients that pluto can't see, and a production checklist as code that runs on kind.

Version note: The examples in this post were run with kind 0.33 and Kubernetes 1.37, kubectl 1.37, pluto 5.24, actionlint 1.7.12, Azure.Identity 1.21 and .NET 10. Nothing was run on cloud clusters; the cloud-side information comes from the providers' documentation as of November 2026.

Guess First

Before upgrading your cluster from Kubernetes 1.37 to the next minor version, you check for deprecated APIs (Application Programming Interfaces). You run pluto detect-api-resources and the answer is clear: "There were no resources found with known deprecated apiVersions." Is the upgrade safe? Looking at the cluster's own metrics in the same minute, you see this line:

apiserver_requested_deprecated_apis{group="",removed_release="",resource="endpoints",subresource="",version="v1"} 1

How can both be true at the same time? The answer is a small example of this post's main idea: being ready for production doesn't end with looking at the objects in the cluster; you also need to see everything that talks to those objects.

Managed Kubernetes: What You Hand Over, What You Keep

AKS (Azure Kubernetes Service), EKS (Amazon Elastic Kubernetes Service) and GKE (Google Kubernetes Engine) run the control plane for you: the API server, etcd, the scheduler, the controller-manager, their redundancy, certificate rotation and patches. The nodes, the timing of node upgrades, the add-ons you install in the cluster and, of course, your app remain your responsibility; modes like GKE Autopilot, EKS Auto Mode and AKS Automatic take over most of node management too, but in return they put stricter rules on the pod spec. A self-managed cluster (kubeadm, k3s, Rancher) only makes sense with an on-prem requirement or a very specific networking need; the operational cost we discussed in the first post of the series multiplies when you run the control plane yourself. For Bookstore the choice is a managed cluster, and for most teams the first setup is a single node pool, the provider's default CNI (Container Network Interface) and managed PostgreSQL. Building the cluster as code is the topic of the upcoming Infrastructure as Code: Terraform series; here we focus on how the app gets to the cluster.

Pulling Images Without Secrets

On the local cluster we loaded images onto the nodes with kind load. In the cloud the image is pulled from a registry, and the best way is to give the node's identity read access to the registry; you don't keep a registry password in a Kubernetes Secret:

  • AKS: az aks update --attach-acr grants the kubelet's managed identity the AcrPull role on the ACR (Azure Container Registry).
  • EKS: the AmazonEC2ContainerRegistryPullOnly policy for ECR (Elastic Container Registry) is attached to the node IAM (Identity and Access Management) role.
  • GKE: images are pulled from Artifact Registry in the same project with default settings; for another project, the node service account gets roles/artifactregistry.reader.

Only for cases like another organization's registry or a private Docker Hub repo do you go back to the imagePullSecrets from the fifteenth post.

Workload Identity: Secretless Cloud Access for the App

In the twelfth post we suggested putting Data Protection keys in blob storage. For the API to reach blob storage and Key Vault it needs an identity, and that identity doesn't have to be a connection string or a client secret. Workload identity federates the pod's ServiceAccount token with the cloud provider's identity system: the short-lived token signed by Kubernetes is exchanged for a short-lived cloud token. On AKS, the ServiceAccount states which Entra ID application it represents, and the pod states that the webhook must kick in:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: api
  namespace: bookstore-prod
  annotations:
    azure.workload.identity/client-id: "00000000-0000-0000-0000-000000000000"
---
# in the Deployment's pod template
metadata:
  labels:
    app.kubernetes.io/name: api
    azure.workload.identity/use: "true"
spec:
  serviceAccountName: api

Careful: in the fifteenth post we set automountServiceAccountToken: false on this ServiceAccount. Workload identity uses a separate projected token mounted by the webhook; still, don't take this combination to production without trying it on a real AKS cluster. On the cluster side, --enable-oidc-issuer --enable-workload-identity is turned on, and a federated credential with the subject system:serviceaccount:bookstore-prod:api is added to the managed identity in Entra ID. On the app side the code change is small:

using Azure.Core;
using Azure.Identity;
using Microsoft.AspNetCore.DataProtection;

var builder = WebApplication.CreateBuilder(args);

// in the pod the webhook injects AZURE_CLIENT_ID, AZURE_TENANT_ID and AZURE_FEDERATED_TOKEN_FILE
TokenCredential credential = new WorkloadIdentityCredential();

builder.Services.AddDataProtection()
    .SetApplicationName("bookstore")
    .PersistKeysToAzureBlobStorage(new Uri(builder.Configuration["DataProtection:BlobUri"]!), credential)
    .ProtectKeysWithAzureKeyVault(new Uri(builder.Configuration["DataProtection:KeyUri"]!), credential);

var app = builder.Build();
app.MapGet("/", () => "Bookstore.Api");
app.Run();

We compiled this code and ran it on a local machine without the variables the webhook would inject; this is exactly the error you'll see when you forget the pod label:

fail: Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider[48]
      Azure.Identity.CredentialUnavailableException: WorkloadIdentityCredential authentication unavailable. The workload options are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/workloadi...

For local development DefaultAzureCredential (which tries environment variables, workload identity and the Azure CLI (Command Line Interface) session in turn) is more convenient; in production we prefer WorkloadIdentityCredential, which makes it explicit which identity is used. On EKS the counterpart is EKS Pod Identity: no annotation on the ServiceAccount; aws eks create-pod-identity-association binds a namespace/ServiceAccount pair to an IAM role, and the AWS (Amazon Web Services) SDK (Software Development Kit) for .NET (at least 3.7.734.0) finds the identity itself through the container credential provider. The AWS documentation recommends Pod Identity over IRSA (IAM Roles for Service Accounts). On GKE, Workload Identity Federation is always on in Autopilot; the recommended path is to grant the IAM role directly to a principal:// identity that represents the Kubernetes ServiceAccount. In all three the result is the same: not a single cloud secret in the repo, the pipeline or the cluster.

CI: Image Push, Migration and Deploy with Kustomize

The workflow from GitHub Actions to AKS combines the pieces of earlier posts in order: signing in to Azure with OIDC (OpenID Connect), building the image and pushing it to ACR, running the migration Job from the thirteenth post, updating the image in the overlay by digest and applying it as in the fourteenth post, and finally running the checklist below:

name: deploy-aks

on:
  push:
    branches: [main]

permissions:
  id-token: write
  contents: read

env:
  REGISTRY: bookstoreacr.azurecr.io
  CLUSTER: bookstore-prod
  RESOURCE_GROUP: rg-bookstore-prod

jobs:
  deploy:
    runs-on: ubuntu-24.04
    environment: production
    steps:
      - uses: actions/checkout@v7

      - uses: azure/login@v3
        with:
          client-id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

      - name: Build and push the API image
        id: api
        run: |
          az acr login --name "${REGISTRY%%.*}"
          image="$REGISTRY/bookstore-api:${GITHUB_SHA::12}"
          docker build -t "$image" src/Bookstore.Api
          docker push "$image"
          digest=$(docker inspect --format '{{index .RepoDigests 0}}' "$image")
          echo "digest=$digest" >> "$GITHUB_OUTPUT"

      - uses: azure/use-kubelogin@v1
        with:
          kubelogin-version: v0.2.22

      - uses: azure/aks-set-context@v5
        with:
          resource-group: ${{ env.RESOURCE_GROUP }}
          cluster-name: ${{ env.CLUSTER }}
          use-kubelogin: "true"

      - name: Run the migration job
        run: ./deploy/run-migration.sh "deploy/jobs/migrate.yaml" "migrate-${GITHUB_SHA::12}"

      - name: Deploy with Kustomize
        working-directory: deploy/overlays/prod
        run: |
          kustomize edit set image "bookstore-api=${{ steps.api.outputs.digest }}"
          kubectl apply -k .
          kubectl -n bookstore-prod rollout status deploy/api --timeout=300s

      - name: Production checklist
        run: ./deploy/prod-check.sh bookstore-prod

This workflow was not run on GitHub or with act: OIDC sign-in, ACR and AKS require a real Azure subscription. We validated it with actionlint 1.7.12 (exit code 0; shellcheck wasn't installed, so the run blocks weren't scanned separately), and checked the actions' input names against their own action.yml files. The run-migration.sh and prod-check.sh scripts in the workflow, however, really ran on kind. On AKS with Entra ID integration and local accounts disabled, kubectl needs kubelogin to get a token; that's what use-kubelogin in the workflow is for. If the runner doesn't have the kustomize binary, install it in a separate step. The Azure side of the OIDC setup is covered in the OIDC deploy-to-Azure post of the GitHub Actions series, and environment approvals in that series' Environments post.

The Upgrade Calendar and Deprecated API Checks

Kubernetes ships three minor versions a year and each minor gets roughly 14 months of patches; community support for 1.35 ends on 28 February 2027 and for 1.37 on 28 October 2027. Providers put their own calendars on top: AKS supports three GA (General Availability) minor versions and extends that to about two years with LTS (Long-Term Support) in the Premium tier; EKS gives 12 more months of extended support after 14 months of standard support, but the price per cluster-hour goes from $0.10 to $0.60; GKE works with release channels (Rapid, Regular, Stable, Extended). The lesson is simple: plan at least one, ideally two upgrades a year; extended support is insurance, not a strategy.

What breaks an upgrade is usually a removed API (the eighteenth post of the series). The answer to the contradiction in Guess First is here. pluto looks at objects stored in the cluster and at manifest files; there were no objects stored with an old apiVersion in our cluster. But someone ran kubectl get endpoints:

$ kubectl -n bookstore-prod get endpoints api
Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
NAME   ENDPOINTS                           AGE
api    10.244.1.37:8080,10.244.2.68:8080   39s

The API server recorded this request in the apiserver_requested_deprecated_apis metric. In production the caller is not a person but an old monitoring agent, an operator, or your own tool using an old client library. The metric's removed_release label tells you in which version the API will be removed; here it's empty, because no removal version has been announced for v1 Endpoints yet. Providers use this information: AKS blocks a minor upgrade if it saw deprecated API usage in the last 12 hours; EKS shows the last 30 days of usage in upgrade insights; GKE pauses automatic minor upgrades when it sees deprecated usage. Your pre-upgrade checklist should include both: pluto for manifests and cluster objects, and the API server metric or the provider's insights for clients.

The Production Checklist as Code

We turned the rules discussed throughout the series into a script that runs in the pipeline instead of a document. For every Deployment it checks probes, resources, the image tag, security settings, the replica count or HPA (Horizontal Pod Autoscaler) and the PDB (PodDisruptionBudget):

#!/usr/bin/env bash
# usage: ./prod-check.sh <namespace>
set -euo pipefail
ns="$1"
fail=0

deployments=$(kubectl -n "$ns" get deploy -o json)
hpas=$(kubectl -n "$ns" get hpa -o json)
pdbs=$(kubectl -n "$ns" get pdb -o json)

for name in $(echo "$deployments" | jq -r '.items[].metadata.name'); do
    d=$(echo "$deployments" | jq --arg n "$name" '.items[] | select(.metadata.name == $n)')
    labels=$(echo "$d" | jq -c '.spec.selector.matchLabels')
    problems=()

    for c in $(echo "$d" | jq -r '.spec.template.spec.containers[].name'); do
        ct=$(echo "$d" | jq --arg c "$c" '.spec.template.spec.containers[] | select(.name == $c)')
        [[ $(echo "$ct" | jq '.readinessProbe != null') == true ]] || problems+=("$c: no readinessProbe")
        [[ $(echo "$ct" | jq '.livenessProbe != null') == true ]] || problems+=("$c: no livenessProbe")
        [[ $(echo "$ct" | jq '.resources.requests.cpu != null and .resources.requests.memory != null') == true ]] \
            || problems+=("$c: missing cpu/memory requests")
        [[ $(echo "$ct" | jq '.resources.limits.memory != null') == true ]] || problems+=("$c: no memory limit")
        [[ $(echo "$ct" | jq -r '.image') =~ :latest$|^[^:]+$ ]] && problems+=("$c: mutable image tag")
        [[ $(echo "$ct" | jq '.securityContext.readOnlyRootFilesystem == true') == true ]] \
            || problems+=("$c: writable root filesystem")
    done

    [[ $(echo "$d" | jq '.spec.template.spec.securityContext.runAsNonRoot == true') == true ]] \
        || problems+=("pod: runAsNonRoot not set")
    [[ $(echo "$d" | jq '.spec.template.spec.automountServiceAccountToken == false') == true ]] \
        || problems+=("pod: service account token mounted")

    has_hpa=$(echo "$hpas" | jq --arg n "$name" '[.items[] | select(.spec.scaleTargetRef.name == $n)] | length')
    replicas=$(echo "$d" | jq '.spec.replicas')
    [[ $has_hpa -gt 0 || $replicas -ge 2 ]] || problems+=("single replica without HPA")

    has_pdb=$(echo "$pdbs" | jq --argjson l "$labels" '[.items[] | select(.spec.selector.matchLabels == $l)] | length')
    [[ $has_pdb -gt 0 ]] || problems+=("no PodDisruptionBudget")

    if [[ ${#problems[@]} -eq 0 ]]; then
        echo "OK    $ns/$name"
    else
        fail=1
        for p in "${problems[@]}"; do echo "FAIL  $ns/$name  $p"; done
    fi
done
exit $fail

Into the bookstore-prod namespace on kind we put the hardened API from the fifteenth post and the Angular web app (nginx-unprivileged, read-only file system, an emptyDir for /tmp, config.json from a ConfigMap). The first run, and the same script's output on the older bookstore namespace:

$ ./prod-check.sh bookstore-prod
FAIL  bookstore-prod/api  api: no livenessProbe
FAIL  bookstore-prod/api  no PodDisruptionBudget
FAIL  bookstore-prod/web  web: no livenessProbe
FAIL  bookstore-prod/web  no PodDisruptionBudget
$ ./prod-check.sh bookstore
FAIL  bookstore/api  api: no livenessProbe
FAIL  bookstore/api  api: writable root filesystem
FAIL  bookstore/api  pod: runAsNonRoot not set
FAIL  bookstore/api  pod: service account token mounted
FAIL  bookstore/api  no PodDisruptionBudget

While focusing on security we forgot the liveness probe and the PDB; that's exactly the value of a checklist. After adding liveness probes, two PDBs and an HPA for the API:

$ ./prod-check.sh bookstore-prod
OK    bookstore-prod/api
OK    bookstore-prod/web

Items the script can't check belong on the list too: graceful shutdown and preStop (tenth post), resource values chosen by measurement (ninth), NetworkPolicy and Pod Security Standards (fifteenth), a tested restore of the database backup (twelfth), log, metric and trace collection with alerts (the upcoming Observability series), the Gateway and TLS (Transport Layer Security) (sixth post and the upcoming SSL (Secure Sockets Layer)/TLS series). In the cloud the Gateway API implementation also depends on the provider: GKE's own Gateway controller, Application Gateway for Containers or the app routing add-on's Gateway API mode on AKS, the AWS Load Balancer Controller on EKS.

Production Notes

Before the first production deploy, do a "day 2" rehearsal: drain a node, delete a pod, roll back a deploy, restore the database from backup into another instance. Each of these is something you tried on kind in a post of this series; you don't want to try them in the cloud for the first time during an incident. For node pool upgrades think about surge settings and PDBs together: a maxUnavailable: 1 PDB and a single-replica Deployment either lock the upgrade or force downtime. Watch cost early: pods with inflated requests, idle nodes and forgotten load balancers are the fastest-growing items on the bill. And automate upgrades but don't leave them unattended: the provider's maintenance window must not collide with your release calendar.

Common Pitfalls

Copying the registry password into every namespace with imagePullSecrets, when the node identity is enough. Keeping a client secret for cloud access in a Kubernetes Secret. Getting the pod label or the namespace/ServiceAccount name in the federated credential's subject wrong for workload identity and losing hours to "workload options are not fully configured". Checking deprecated APIs only through manifests and missing the clients that talk to the cluster with old APIs. Treating extended support as a permanent solution and paying six times the control plane fee. Keeping the checklist on a wiki page instead of in the pipeline. And making the first deploy straight to production because everything works on the local cluster, without testing the networking, identity and load balancer differences in the cloud.

A Small Experiment

★☆☆ Add two rules to prod-check.sh: the image must be specified by digest (@sha256:), and terminationGracePeriodSeconds must be greater than the app's Shutdown:TimeoutSeconds. Run it against the namespaces on kind.

★★☆ On the kind cluster, call kubectl get endpoints and kubectl get --raw /apis/discovery.k8s.io/v1/endpointslices, then read the apiserver_requested_deprecated_apis metric. Which call shows up in the metric? If your own .NET app has code that reads Endpoints with the KubernetesClient package, where would you see the same warning?

★★★ Open a small AKS (or EKS/GKE) cluster with a free trial account, set up the API with workload identity so it writes Data Protection keys to a blob container, and run this post's workflow in your own repo. Write down everything that works on kind but not in the cloud; that list is your real production checklist. Don't forget to delete the cluster when you're done.

With this post, the path that takes the app to production is complete. In the last block of the series we return to versions: in the next post we read the 2018–2021-era manifests still circulating on the internet, dockershim and PodSecurityPolicy, and translate them into their current equivalents.