Mert Özen Further With Every Line
Security DevSecOps — #05

Container and IaC Scanning: Image Vulnerabilities, Base Image Choice and Manifest Scanning

Mert Özen Nov 14, 2026 13 min 4 views
Container and IaC Scanning: Image Vulnerabilities, Base Image Choice and Manifest Scanning

Image scanning checks the operating system packages and the application dependencies of the artifact you actually deploy; most findings come from unused packages, and a chiseled, non-root base image removes most of them. With real output we show the Bookstore image dropping from 42 findings to 6, how Trivy and Grype differ, verifying the scanner with a SHA pin and a cosign signature after the March 2026 Trivy incident, and Dockerfile, Kubernetes and Terraform scanning.

Version note: The code in this post was built and run with .NET 10; the images are based on mcr.microsoft.com/dotnet/aspnet:10.0 (Ubuntu 24.04), and the scans were run with Trivy 0.76.2, Grype 0.121.3, hadolint 2.15.1 and cosign 3.1.4 against vulnerability databases dated 13–14 November 2026.

Guess First

The Bookstore API image has come out of the same Dockerfile for years: the full aspnet base image, curl installed for the health check, and the root user. The code itself has no known vulnerabilities; NuGetAudit is clean. Guess: how many vulnerabilities does Trivy find in this image? What happens to that number when you switch the base image to a chiseled one? And one more question: if the action tag of the scanner in your pipeline were pointed to a different commit overnight, would you notice?

In this post we scan container images and IaC files: the difference between operating system packages and application dependencies in an image, how chiseled images shrink the attack surface, why two scanners give different numbers for the same image, Dockerfile and Kubernetes manifest checks, and the lesson of the March 2026 Trivy incident: the scanner itself is part of the supply chain.

What Is in the Image: Operating System Packages and Application Dependencies

This is the image's Dockerfile (the SDK stage uses BUILDPLATFORM for cross-compiling; the details of Dockerfile design are the topic of the Docker series):

FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:10.0 AS build
ARG TARGETARCH
WORKDIR /src
COPY . .
RUN dotnet publish src/Bookstore.Api -c Release -a $TARGETARCH -o /app

FROM mcr.microsoft.com/dotnet/aspnet:10.0
RUN apt-get update && apt-get install -y curl
WORKDIR /app
COPY --from=build /app .
HEALTHCHECK CMD curl -f http://localhost:8080/health || exit 1
ENTRYPOINT ["dotnet", "Bookstore.Api.dll"]
docker build --platform linux/amd64 -f Dockerfile.v1 -t bookstore-api:v1 .
trivy image --scanners vuln bookstore-api:v1
bookstore-api:v1 (ubuntu 24.04)
Total: 42 (UNKNOWN: 0, LOW: 5, MEDIUM: 37, HIGH: 0, CRITICAL: 0)

All forty-two findings are in Ubuntu packages: libc6, libgnutls30t64, tar, dash and the Kerberos libraries, for example. The application's deps.json and the .NET runtime's own deps.json files were scanned too and came out clean. An image scanner looks at two separate things: operating system packages, read from the package manager's database, and application dependencies, read from the files of language ecosystems. To see the second part, I built the same application with the old SqlClient 5.1.1:

app/Bookstore.Api.deps.json (dotnet-core)
Total: 6 (UNKNOWN: 0, LOW: 0, MEDIUM: 4, HIGH: 2, CRITICAL: 0)

These are the same advisories NuGetAudit found in the third post. So if we already catch them at restore, why scan again in the image? Because the image is the real artifact that gets deployed; a publish setting, a multi-stage mistake or a DLL copied from another project can bring into the image something restore never saw. Image scanning is the final answer to "what are we actually running".

Choosing a Base Image: Chiseled and Non-root

Most of the 42 findings come from packages the application never uses. Microsoft's chiseled images contain only the parts of Ubuntu that .NET needs to run: no shell, no package manager, and a non-root user defined by default. The same application's second Dockerfile:

FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:10.0 AS build
ARG TARGETARCH
WORKDIR /src
COPY . .
RUN dotnet publish src/Bookstore.Api -c Release -a $TARGETARCH -o /app

FROM mcr.microsoft.com/dotnet/aspnet:10.0-noble-chiseled
WORKDIR /app
COPY --from=build /app .
USER $APP_UID
ENTRYPOINT ["dotnet", "Bookstore.Api.dll"]
ImageSizeUserOS findingsWith a fix
v1: aspnet:10.0 + curl141 MBroot (uid=0)42 (37 medium, 5 low)13
v2: aspnet:10.0-noble-chiseled58.7 MB16546 (medium)6

Findings dropped from 42 to 6, size from 141 MB to 58.7 MB. In v1, docker run ... id returned uid=0(root); in v2 an attempt to run sh was rejected with "executable file not found", because the image has no shell. I started the container and sent a request to /covers/1; the application listened on 8080 and returned 404, so the application runs normally in the non-root, shell-less image. A non-root user reduces what an attacker holds in container escape scenarios; the missing shell removes tools that could be used in an RCE situation.

Two warnings. First, chiseled images set DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=true; there is no ICU. For an application like Bookstore that needs Turkish sorting and culture-specific formatting, choose the -noble-chiseled-extra variant, which includes ICU and time zone data. Second, all 6 remaining findings are in libc6 2.39-0ubuntu8.9, with the fix in 2.39-0ubuntu8.12. Since the chiseled image has no apt, you cannot upgrade it inside the image; the fix arrives with a new version of the base image. That is why rebuilding the image regularly matters as much as choosing the base image.

Trivy and Grype: Same Image, Different Numbers

I scanned the same v2 image with Grype:

grype bookstore-api:v2 --platform linux/amd64
NAME        INSTALLED           FIXED IN          TYPE  VULNERABILITY    SEVERITY    EPSS          RISK
libc6       2.39-0ubuntu8.9                       deb   CVE-2026-100290  Medium      0.3% (27th)   0.2
libc6       2.39-0ubuntu8.9     2.39-0ubuntu8.12  deb   CVE-2026-8674    Medium      0.3% (24th)   0.2
libc6       2.39-0ubuntu8.9     2.39-0ubuntu8.12  deb   CVE-2026-84243   Medium      0.3% (22nd)   0.1
libc6       2.39-0ubuntu8.9     2.39-0ubuntu8.12  deb   CVE-2026-97399   Medium      0.3% (20th)   0.1
libc6       2.39-0ubuntu8.9                       deb   CVE-2026-115819  Medium      0.3% (18th)   0.1
libc6       2.39-0ubuntu8.9     (won't fix)       deb   CVE-2016-20013   Negligible  2.2% (82nd)   0.1
libssl3t64  3.0.13-0ubuntu3.16                    deb   CVE-2026-95692   Medium      0.2% (11th)   0.1
openssl     3.0.13-0ubuntu3.16                    deb   CVE-2026-95692   Medium      0.2% (11th)   0.1
...

Grype gave 19 matches: 6 with a fix (the same 6 findings Trivy reported), 12 without a fix yet, and 1 that the distribution marked as "won't fix". With --only-fixed Grype also drops to the same 6 findings. The difference lies in how the scanners interpret Ubuntu's security data and whether they show records without a fix by default. Grype also shows an EPSS column and a risk score, which directly supports the prioritization from the third post.

The lesson: do not treat scanner numbers as absolute truth; a difference between two scanners for the same image is normal. Pick one scanner as the gate in the pipeline and know its behaviour well; use the second for a periodic comparison. Building the gate on "critical and high with a fix" neutralizes most of these differences: in v1, --ignore-unfixed takes 42 findings down to 13, and --severity CRITICAL,HIGH --ignore-unfixed --exit-code 1 returns 0 for both images. The image with SqlClient 5.1.1, however, returned 1 on the same gate.

The Scanner Is Supply Chain Too: The March 2026 Trivy Incident

According to Aqua Security's incident write-up, attackers obtained a privileged token from a misconfiguration in Trivy's GitHub Actions environment at the end of February 2026; the credential rotation on 1 March was incomplete. On 19 March 2026, 76 of the 77 version tags of aquasecurity/trivy-action and all tags of setup-trivy were redirected to malicious commits, and a malicious Trivy binary (0.69.4) was published the same day; on 22 March malicious 0.69.5 and 0.69.6 images were pushed to Docker Hub. The malicious action collected the runner's secrets and sent them out before running the real scan, so the workflows looked normal. Aqua's recommendations included moving to safe versions, rotating all secrets that the affected pipelines could access, and pinning actions to a full commit SHA instead of a tag.

Three rules follow for Bookstore. First, pin the action by SHA, not by tag: a tag can be moved, a commit SHA cannot. The mechanics are in the GitHub Actions series post "GitHub Actions Security: SHA Pinning, Lockfiles, pull_request_target and Preventing Script Injection". Second, if you download a binary, verify its signature. Since 0.68.1 Trivy publishes a Sigstore bundle (.sigstore.json) for every release asset; verifying it with cosign checks that the binary came out of Trivy's own release workflow:

cosign verify-blob trivy_0.76.2_Linux-64bit.tar.gz \
  --bundle trivy_0.76.2_Linux-64bit.tar.gz.sigstore.json \
  --certificate-oidc-issuer=https://token.actions.githubusercontent.com \
  --certificate-identity "https://github.com/aquasecurity/trivy/.github/workflows/reusable-release.yaml@refs/tags/v0.76.2"
Verified OK

I ran the same command with the expected identity written as a different repo; cosign returned 1 with failed to verify certificate identity: no matching CertificateIdentity found. Verifying only a checksum is not enough on its own, because you download the checksum file and the binary from the same place; a compromised release can change both. A signature, on the other hand, is bound to a specific workflow identity. We look at how keyless signing works in detail in the eighth post of the series. Third, do not give the scanning job deploy credentials: the scanner needs the image and read access; secrets cannot be stolen from a job that cannot reach them.

name: image

on:
  push:
    branches: [main]

permissions:
  contents: read

jobs:
  image:
    runs-on: ubuntu-latest
    env:
      TRIVY_VERSION: 0.76.2
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
      - uses: sigstore/cosign-installer@756fe2d284c7292c4e9b62c8a28f1e5629c59532 # v4.1.3
      - name: Install and verify Trivy
        run: |
          base="https://github.com/aquasecurity/trivy/releases/download/v${TRIVY_VERSION}"
          file="trivy_${TRIVY_VERSION}_Linux-64bit.tar.gz"
          curl -sSfL -o "$file" "$base/$file"
          curl -sSfL -o "$file.sigstore.json" "$base/$file.sigstore.json"
          cosign verify-blob "$file" --bundle "$file.sigstore.json" \
            --certificate-oidc-issuer=https://token.actions.githubusercontent.com \
            --certificate-identity "https://github.com/aquasecurity/trivy/.github/workflows/reusable-release.yaml@refs/tags/v${TRIVY_VERSION}"
          tar -xzf "$file" trivy
      - run: docker build -t "bookstore-api:$GITHUB_SHA" .
      # fixable critical/high fails the build; the rest is reported
      - run: ./trivy image --exit-code 1 --severity CRITICAL,HIGH --ignore-unfixed "bookstore-api:$GITHUB_SHA"
      - run: ./trivy config --exit-code 1 --severity CRITICAL,HIGH --ignorefile .trivyignore.yaml deploy

I validated the workflow with actionlint; I ran the verification command locally on the Linux asset, and the two scan commands against the Bookstore image and the deploy folder. If you prefer using the action, pin aquasecurity/trivy-action to a full SHA as well; at the time of writing the latest version was v0.36.0.

IaC: Dockerfile, Kubernetes Manifest and Terraform

The files that build the image are code too. The Dockerfile linter hadolint found four rule violations in v1:

Dockerfile.v1:8 DL3008 warning: Pin versions in apt get install. Instead of `apt-get install <package>` use `apt-get install <package>=<version>`
Dockerfile.v1:8 DL3009 info: Delete the apt lists (/var/lib/apt/lists) after installing something
Dockerfile.v1:8 DL3015 info: Avoid additional packages by specifying `--no-install-recommends`
Dockerfile.v1:11 DL3025 warning: Use arguments JSON notation for CMD and ENTRYPOINT arguments

Trivy's config scan found two of 27 checks failing in the same Dockerfile, both high: DS-0002 (no non-root USER) and DS-0029 (missing --no-install-recommends). In v2 the only finding is DS-0026 (low): no HEALTHCHECK. Since Bookstore runs on Kubernetes, and Kubernetes uses its own probes rather than the Dockerfile's HEALTHCHECK, we accept this finding deliberately. hadolint also left an info note (DL3066) on the USER $APP_UID line in v2; running hadolint with --failure-threshold warning keeps that line from failing the build.

The Kubernetes manifest is more striking. The first deployment contained no security settings except a privileged: true someone had added to "fix a file permission problem", and Trivy gave 18 findings; among them, as high, KSV-0017 (privileged), KSV-0014 (writable root file system) and KSV-0118 (default security context). We fixed it with the restricted profile settings from the Kubernetes series post that covers RBAC and securityContext, "Kubernetes Security: ServiceAccounts, RBAC, Pod Security Standards, securityContext and NetworkPolicy":

apiVersion: apps/v1
kind: Deployment
metadata:
  name: bookstore-api
  namespace: bookstore
spec:
  replicas: 2
  selector:
    matchLabels:
      app: bookstore-api
  template:
    metadata:
      labels:
        app: bookstore-api
    spec:
      securityContext:
        runAsNonRoot: true
        runAsUser: 1654
        runAsGroup: 1654
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: api
          image: ghcr.io/bookstore/bookstore-api:1.4.0
          ports:
            - containerPort: 8080
          resources:
            requests:
              cpu: 100m
              memory: 256Mi
            limits:
              cpu: 500m
              memory: 512Mi
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop: ["ALL"]
          volumeMounts:
            - name: tmp
              mountPath: /tmp
      volumes:
        - name: tmp
          emptyDir: {}

The findings dropped from 18 to 3: two low (KSV-0020 and KSV-0021, recommending a UID and GID above 10000) and one medium (KSV-0125, images should come from trusted registries). For readOnlyRootFilesystem we mounted an emptyDir at /tmp; ASP.NET Core can write large request bodies and some temporary files to /tmp.

A quick look at the Terraform side. An Azure storage account definition for the cover images had been written with min_tls_version = "TLS1_0"; Trivy gave six findings, two of them critical: TLS 1.0 and default access left open in the network rules. After adding min_tls_version = "TLS1_2", https_traffic_only_enabled = true and a network_rules block with default_action = "Deny", AZU-0010 (bypass access for Azure services) appeared as high; with bypass = ["AzureServices"] no critical or high findings remain, and two medium and one low finding for logging, geo-redundancy and infrastructure encryption stay in the report. I did not try the Terraform file with terraform plan, I only scanned it with Trivy; Terraform itself is the topic of the Infrastructure as Code: Terraform series.

Exceptions: .trivyignore.yaml with a Reason and an Expiry

We made a decision about the three remaining Kubernetes findings: UID 1654 is the chiseled image's own app user, and we use it as is rather than remapping it; the registry restriction we will enforce at the cluster level with an admission policy in the eighth post. Trivy's YAML ignore file can take a reason (statement) and an expiry date (expired_at) for each exception; the Trivy documentation marks this format as experimental:

misconfigurations:
  - id: KSV-0020
    paths: ["k8s/deployment.yaml"]
    statement: The chiseled image's app user is UID 1654; we keep it instead of remapping.
  - id: KSV-0021
    paths: ["k8s/deployment.yaml"]
    statement: Same as KSV-0020 for the app group.
  - id: KSV-0125
    statement: Admission policy restricts registries to ghcr.io/bookstore (post 8).
    expired_at: 2027-03-31

I ran two tests. When I moved expired_at to 2026-11-01, the KSV-0125 finding came back; an expired exception does not linger silently. The second was a trap: when I ran the scan against the deploy/k8s folder, KSV-0020 and KSV-0021 were not suppressed, because paths are relative to the scanned target and in that scan the file's path was deployment.yaml. Running the scan always from the same root (deploy) applied all three exceptions correctly, and the Kubernetes manifest passed with zero findings.

At Build Time or in the Registry? Vulnerabilities Without a Fix

A build-time scan uses that day's vulnerability database. An image that passes cleanly today will still be running in production when tomorrow's CVE is published. That is why you need two scans: a gate at build time that keeps new vulnerabilities out of production, and a job that scans deployed images regularly. For the second, you can use a cloud registry's own scanning features or a scheduled job that scans the deployed image digests every day. A lighter way is to keep the image's SBOM instead of the image and scan that:

trivy image --format cyclonedx -o bookstore-api.cdx.json bookstore-api:legacy
trivy sbom --severity HIGH,CRITICAL bookstore-api.cdx.json
Total: 2 (HIGH: 2, CRITICAL: 0)

Without pulling the image again, the same two high findings (Azure.Identity and SqlClient) came through the SBOM. We set up producing, storing and querying SBOMs in detail in the seventh post of the series.

Living with vulnerabilities that have no fix is part of the job too. In the Grype output for v2, 12 findings have no fix yet. Failing the gate on them gains nothing, because there is nothing the developer can do; but ignoring them is not right either. The right approach is to track the finding, take the fix with a base image update when it is published, and turn the cases where you can say "this does not affect us, because..." into a machine-readable statement. That statement is called VEX, and it is the topic of the seventh post.

In Production

Pinning the base image by digest gives reproducibility but leaves updating to you. If you use a tag, rebuild the image regularly; if you use a digest, let a bot such as Dependabot or Renovate update the base image digest. Either way, the goal is for a fix in the base image to reach production within days.

Treat the scanner database as a dependency too. Trivy and Grype download the vulnerability database before scanning and keep it in a cache; on runners with restricted network access this step can fail silently and lead to a scan with an old database. Check the database date in the pipeline log; the database build time in Grype's JSON (JavaScript Object Notation) output is handy for this.

If you scan the same image in more than one job, uploading the results to code scanning as SARIF lets you see them in one place and use the same triage flow as the SAST findings from the fourth post.

Common Pitfalls

First, using an action by tag. The March 2026 incident showed concretely that tags can be moved; pin by SHA and take updates through a PR.

Second, verifying a binary only with a checksum. If the checksum and the binary come from the same source, a compromised release changes both; signature verification also checks the expected workflow identity.

Third, switching to a chiseled image and forgetting the globalization setting. In invariant mode Turkish culture rules do not work; if you need ICU, use the -extra variant.

Fourth, treating "the image is clean" as permanent. Today's scan used today's database; images in the registry must be rescanned regularly.

Fifth, silencing Kubernetes findings with shortcuts like privileged: true. Solving a permission problem with a privileged container turns the problem into a security hole.

A Small Experiment

★☆☆ Scan your project's image with Trivy and split the findings into operating system packages and application dependencies. How many have a fix?

★★☆ Build the same application with aspnet:10.0 and aspnet:10.0-noble-chiseled-extra; compare size, user and number of findings, then verify with a request that the application really runs in the chiseled image.

★★★ List every scanner action and every downloaded binary in your pipeline. Which are used by tag, and which have their signatures verified? Pin one to a SHA, add a cosign verify-blob step for one binary, and see the step fail with a wrong identity.

We scanned images and infrastructure files statically; but some weaknesses only show up while the application is running: a missing header, a wrong cookie flag, information leaked by an error page. In the next post we move on to DAST and scan the running Bookstore with ZAP.