.NET CI with GitHub Actions: setup-dotnet, global.json, NuGet Cache, Test Reports and Coverage
In a .NET 10 CI, pin the SDK with global.json and setup-dotnet@v6, restore with lock files and --locked-mode, key the NuGet cache to the lock hash, and gate with dotnet format and -warnaserror. We produce TRX and Cobertura reports with xUnit v3 on Microsoft.Testing.Platform, write them to the job summary and keep them with upload-artifact, and show act's v7 artifact limit and the Linux file-name pitfall with real output.
Version note: The examples in this post were tested with act 0.2.97, actionlint 1.7.12, Docker 29.2, .NET 10 (SDK 10.0.403), xUnit v3 4.0, Microsoft.Testing.Extensions.CodeCoverage 18.12, actions/setup-dotnet@v6 and actions/upload-artifact@v7. The workflows were run locally with act; they were not run on GitHub.
Guess First
Alan is adapting a .NET CI (Continuous Integration) setup copied from an old project to Bookstore. The tests are written with the current xUnit v3 release, and the command in the workflow is the line he has used for years:
dotnet test api/tests/Bookstore.Api.TestsHis machine has the .NET 10 SDK (Software Development Kit) installed and the project targets net10.0. The repository's global.json contains only the SDK version. What does this command do: run the tests, print a warning, or never start?
$ dotnet test api/tests/Bookstore.Api.Tests
~/.nuget/packages/microsoft.testing.platform.msbuild/2.5.1/buildMultiTargeting/Microsoft.Testing.Platform.MSBuild.targets(395,5): error : Testing with VSTest target is no longer supported by Microsoft.Testing.Platform on .NET 10 SDK and later. If you use dotnet test, you should opt-in to the new dotnet test experience. For more information, see https://aka.ms/dotnet-test-mtp-errorIt never starts. xUnit v3 4.x test projects run on Microsoft.Testing.Platform (MTP), and in the .NET 10 SDK running MTP projects through the old VSTest path is no longer supported. The fix is a single line, but you need to know which line; in this post we look at it and at the rest of a .NET CI: pinning the SDK, lock files, the NuGet cache, quality gates, test reports, coverage and artifacts.
Pinning the SDK: global.json and setup-dotnet@v6
The final form of the global.json we have used since the first post:
{
"sdk": {
"version": "10.0.400",
"rollForward": "latestFeature"
},
"test": {
"runner": "Microsoft.Testing.Platform"
}
}The sdk section says two things: at least 10.0.400, and the newest feature band within the same major/minor. actions/setup-dotnet@v6 reads this file through the global-json-file input; its README states that for latestPatch, latestFeature, latestMinor and latestMajor it installs the highest matching SDK, and that with rollForward the version must be a full SDK version (no wildcards like 10.0.*). In the act runs this rule picked 10.0.403; this machine has 10.0.402 installed, which satisfies the rule too. The difference is small, but it is worth knowing it exists: CI always gets the newest patch, a developer machine may be one step behind.
The test section is the fix for the error in the guess section. With .NET 10, dotnet test split into two modes: VSTest and Microsoft.Testing.Platform. In MTP mode dotnet test runs test projects directly as executables; that is why our test projects have <OutputType>Exe</OutputType> and no longer need the Microsoft.NET.Test.Sdk and xunit.runner.visualstudio packages. The command line changed a little too: the project or solution is passed with --project and --solution, and extra arguments for the test framework (reports, coverage) go on the same line.
Lock Files, --locked-mode and the NuGet Cache
Bookstore's api/Directory.Build.props file contains <RestorePackagesWithLockFile>true</RestorePackagesWithLockFile>; a packages.lock.json is generated for every project and committed to the repository. The lock file records the exact version and hash of every package, transitive dependencies included. In CI we restore with --locked-mode: if the lock file and the project files disagree, restore stops. When someone changes a package version in a csproj and forgets to update the lock file:
$ dotnet restore api/Bookstore.slnx --locked-mode
…/api/src/Bookstore.Api/Bookstore.Api.csproj : error NU1004: The package reference Npgsql.EntityFrameworkCore.PostgreSQL version has changed from [10.0.3, ) to [10.0.2, ).The packages lock file is inconsistent with the project dependencies so restore can't be run in locked mode. Disable the RestoreLockedMode MSBuild property or pass an explicit `--force-evaluate` option to run restore to update the lock file.That error is a good thing: CI cannot build with a package tree different from the one resolved on the developer's machine. In the Web Security (OWASP Top 10) series post on OWASP A03 (Software Supply Chain Failures) we discussed the supply chain side of lock files; here we see the reproducible build side.
A side benefit of lock files is that they make cache keys. setup-dotnet's cache input caches the NuGet global packages folder and derives the key from the hash of the lock files; according to the README it fails if it cannot find a lock file:
- uses: actions/setup-dotnet@v6
with:
global-json-file: global.json
cache: true
cache-dependency-path: api/**/packages.lock.json
- run: dotnet restore api/Bookstore.slnx --locked-modeWith act's local cache server there was no cache in the first job and it was saved at the end of that job; later jobs restored it with the same key:
[api-ci/format] | Dotnet cache is not found
[api-ci/format] ⚙ ::set-output:: cache-hit=false
[api-ci/format] ✅ Success - Main dotnet restore api/Bookstore.slnx --locked-mode [5.13875925s]
[api-ci/format] | Cache Size: ~72 MB (75874143 B)
[api-ci/format] | Cache saved with the key: dotnet-cache-Linux-ef61870070da36c522f7ec79330dcf2777402de8d84627544eeb2d86afdb55b8
...
[api-ci/build-test ] | Cache restored from key: dotnet-cache-linux-ef61870070da36c522f7ec79330dcf2777402de8d84627544eeb2d86afdb55b8
[api-ci/build-test ] ⚙ ::set-output:: cache-hit=true
[api-ci/build-test ] ✅ Success - Main Restore [2.025406375s]The cold restore took 5.1 seconds, the restore from cache 2 seconds. In this small project the difference is a few seconds; in a solution with hundreds of packages it is minutes. The hash at the end of the key stays the same as long as the lock files don't change; when a package is updated, a new key is produced and the first job fills the cache again. We will cover how the cache works, eviction and the repository cache limit separately in the debugging and cost post.
Quality Gates: dotnet format and Warnings as Errors
The first job of api-ci.yml is format: after restore it runs dotnet format --verify-no-changes. This command does not change code; if something should change according to the .editorconfig rules, it reports an error and returns a non-zero exit code. The act output on a branch where we left one line indented two spaces short:
[api-ci/format] ⭐ Run Main dotnet format api/Bookstore.slnx --verify-no-changes --no-restore
[api-ci/format] | …/api/src/Bookstore.Api/OrderPricing.cs(16,7): error WHITESPACE: Fix whitespace formatting. Insert '\s\s'. […/api/src/Bookstore.Api/Bookstore.Api.csproj]
[api-ci/format] ❌ Failure - Main dotnet format api/Bookstore.slnx --verify-no-changes --no-restore [3.2338215s]
[api-ci/format] exitcode '2': failure
[api-ci/format] 🏁 Job failedThe message says what is missing on which line of which file; the fix is running dotnet format api/Bookstore.slnx locally. When the format job breaks, all other jobs are skipped because of needs; that is why we put this check first. One note: on this machine the messages of dotnet format came out in Turkish, following the operating system's language; the English output in CI comes from the runner's locale.
In the build step we use -warnaserror. Locally warnings stop nobody and hundreds pile up over time; treating every warning as an error in CI cuts that off from the start. After adding an unused variable:
[api-ci/format] 🏁 Job succeeded
[api-ci/build-test] ⭐ Run Main Build
[api-ci/build-test] | …/api/src/Bookstore.Api/OrderPricing.cs(23,13): error CS0219: The variable 'unused' is assigned but its value is never used […/api/src/Bookstore.Api/Bookstore.Api.csproj]
[api-ci/build-test] | 1 Error(s)
[api-ci/build-test] ❌ Failure - Main Build [1.863064s]
[api-ci/build-test] 🏁 Job failedNote: the format job let this change through, because an unused variable is not a formatting problem but a compiler warning (CS0219). The two gates catch different things; one does not replace the other.
The reason we pass it on the command line instead of writing TreatWarningsAsErrors into Directory.Build.props is the local development experience: a half-written variable should not break the build while you are coding, but everything going to main should go with zero warnings. Both approaches are common; what matters is that the rule is enforced in CI.
Test Reports and Coverage: TRX, Cobertura and the Job Summary
The unit test step makes MTP produce two reports: xUnit v3's TRX (Visual Studio Test Results) report and a coverage report in Cobertura format from Microsoft's code coverage extension. For coverage we added the Microsoft.Testing.Extensions.CodeCoverage package to the test projects:
- name: Unit tests
run: >
dotnet test --project api/tests/Bookstore.Api.Tests --no-build -c Release
--results-directory TestResults --report-xunit-trx
--coverage --coverage-output-format cobertura --coverage-output coverage.cobertura.xml
- name: Test summary
if: ${{ !cancelled() }}
run: python3 .github/scripts/test-summary.py TestResults | tee -a "$GITHUB_STEP_SUMMARY"test-summary.py is the TRX script from the expressions post with a coverage line added:
# Writes a Markdown job summary from TRX test reports and a Cobertura coverage file
import glob
import sys
import xml.etree.ElementTree as ET
results = sys.argv[1]
ns = {"t": "http://microsoft.com/schemas/VisualStudio/TeamTest/2010"}
print("| Tests | Total | Passed | Failed | Skipped |")
print("|---|---:|---:|---:|---:|")
failures = []
for path in sorted(glob.glob(f"{results}/**/*.trx", recursive=True)):
root = ET.parse(path).getroot()
c = root.find("t:ResultSummary/t:Counters", ns).attrib
name = root.find("t:TestDefinitions/t:UnitTest", ns).attrib["storage"].split("/")[-1]
skipped = int(c["total"]) - int(c["executed"])
print(f"| {name} | {c['total']} | {c['passed']} | {c['failed']} | {skipped} |")
failures += [r.attrib["testName"] for r in root.findall("t:Results/t:UnitTestResult[@outcome='Failed']", ns)]
for name in failures:
print(f"\n:x: `{name}`")
for path in sorted(glob.glob(f"{results}/**/*.cobertura.xml", recursive=True)):
cov = ET.parse(path).getroot().attrib
print(f"\nLine coverage: **{float(cov['line-rate']):.1%}** ({cov['lines-covered']}/{cov['lines-valid']}), "
f"branch coverage: **{float(cov['branch-rate']):.1%}**")[api-ci/build-test ] ⭐ Run Main Test summary
[api-ci/build-test ] | | Tests | Total | Passed | Failed | Skipped |
[api-ci/build-test ] | |---|---:|---:|---:|---:|
[api-ci/build-test ] | | Bookstore.Api.Tests.dll | 6 | 6 | 0 | 0 |
[api-ci/build-test ] |
[api-ci/build-test ] | Line coverage: **18.0%** (11/61), branch coverage: **30.8%**18 percent looks low, but it has to be read correctly: the unit tests only test OrderPricing; the endpoints in Program.cs and the DbContext are the integration tests' territory. Use coverage as a map, not a threshold: it answers "which critical code is never tested?", not "are the tests enough?". We chose Cobertura because almost every reporting tool can read it.
Test Results and Publish Output with upload-artifact@v7
The job summary is for the overview; to keep the full TRX and coverage files downloadable we use artifacts. The publish job also uploads the dotnet publish output as an artifact; we will pick that artifact up in the deploy posts:
- uses: actions/upload-artifact@v7
if: ${{ !cancelled() }}
with:
name: test-results
path: TestResults/
retention-days: 14 publish:
needs: [build-test, integration-test]
if: ${{ github.event_name == 'push' }}
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v7
- uses: actions/setup-dotnet@v6
with:
global-json-file: global.json
cache: true
cache-dependency-path: api/**/packages.lock.json
- run: dotnet restore api/src/Bookstore.Api --locked-mode
- run: dotnet publish api/src/Bookstore.Api -c Release --no-restore -o publish
- uses: actions/upload-artifact@v7
with:
name: api-publish
path: publish/
retention-days: 7Running this in act we hit an obstacle at this step: the local artifact server of act 0.2.97 could not talk to upload-artifact@v7. The same step failed after five attempts:
[api-ci/build-test ] ⭐ Run Main actions/upload-artifact@v7
[api-ci/build-test ] | With the provided path, there will be 2 files uploaded
[api-ci/build-test ] | Artifact name is valid!
[api-ci/build-test ] | Root directory input is valid!
[api-ci/build-test ] | Attempt 1 of 5 failed with error: Unexpected end of JSON input. Retrying request in 8000 ms...
...
[api-ci/build-test ] ❌ Failure - Main actions/upload-artifact@v7 [1m29.955726834s]We confirmed it with a separate small workflow: on the same act version upload-artifact@v4 works and @v7 does not. So, for the local run only, we replaced @v7 with @v4 in a copy of the workflow; the workflow in this post and in the repository uses @v7 and was not run with @v7 on GitHub. The act output with @v4:
[api-ci/build-test ] | Artifact test-results has been successfully uploaded! Final size is 3568 bytes. Artifact ID is 3486992502
...
[api-ci/publish ] ✅ Success - Main dotnet publish api/src/Bookstore.Api -c Release --no-restore -o publish [1.566237917s]
[api-ci/publish ] | Artifact api-publish has been successfully uploaded! Final size is 2513485 bytes. Artifact ID is 533688365retention-days sets how long the artifact is kept; it cannot exceed the limit in the repository settings. We kept test results for two weeks and the publish output for one; real releases should live in a GitHub Release or a registry, not in artifacts. The if: ${{ github.event_name == 'push' }} on the publish job skips the publish step on pull requests; artifact storage also counts against your quota.
Production Notes
Most of this workflow's total time in act comes from SDK installation and restore: every job downloads the SDK again, because every job is a separate machine. On GitHub-hosted runners some SDK versions come preinstalled in the image; with check-latest: false you can tell setup-dotnet "if a matching SDK is installed, use it", giving up the guarantee of the newest patch in exchange. The effect of the NuGet cache grows with the project too; with our 72 MB cache, restore in act went from 5.1 seconds down to 2.
There are two ways to do integration tests. In this series we used a service container: the workflow brings the database up and the test code only knows the connection string. The other way is Testcontainers: the test code connects to Docker and starts its own container; the same test runs locally and in CI with no extra setup. The price is needing Docker on the runner and a slightly longer test start. The Entity Framework Core (EF Core) series post "Testing with EF Core: SQLite In-Memory and InMemory Provider Pitfalls" and the ASP.NET Core Web API (Application Programming Interface) series integration test post touch on these options; since Docker is ready on GitHub's Linux runners, both ways work. We did not run a Testcontainers version in this post.
Finally, think before turning coverage into a gate. A rule like "break the build if coverage drops below 80%" can lead to meaningless tests being written. A more useful rule is usually "coverage of new code must not drop", shown as a pull request comment.
Common Pitfalls
File name case on Linux. A test read the file TestData/Books.json as books.json. On macOS (a case-insensitive file system by default) the tests passed; in act's Linux container they broke:
[api-ci/build-test] ⭐ Run Main Unit tests
[api-ci/build-test] | failed Bookstore.Api.Tests.SampleOrderTests.Sample_order_from_file_ships_free (7ms)
[api-ci/build-test] | Xunit.MicrosoftTestingPlatform.XunitException: System.IO.FileNotFoundException : Could not find file '…/api/tests/Bookstore.Api.Tests/bin/Release/net10.0/TestData/books.json'.
[api-ci/build-test] | Test run summary: Failed!
[api-ci/build-test] | total: 7
[api-ci/build-test] | failed: 1
[api-ci/build-test] | succeeded: 6
[api-ci/build-test] ❌ Failure - Main Unit tests [1.236229959s]Every team that develops on Windows or macOS and tests on Linux hits this at least once. Write file names in code exactly as they are on disk; CI running on Linux is an advantage here.
The SDK roll-forward surprise. Without a global.json, .NET uses the newest SDK installed on the machine. That is why a build can start running with a different compiler, without any change, when the runner image is updated and a new SDK arrives. The global.json + setup-dotnet pair keeps this under control.
--no-build with a different configuration. If the build was done with -c Release, the tests must also run with -c Release; otherwise --no-build looks for the Debug output and does not find it.
Adding the lock file to .gitignore. packages.lock.json must be in the repository; without it --locked-mode and the setup-dotnet cache do not work.
Using artifacts for everything. Moving a bin folder between jobs is often slower than rebuilding; artifacts are for lasting results (reports, publish output).
A Small Experiment
★☆☆ Delete the test section from global.json and see what the dotnet test --project ... command says. Then add it back.
★★☆ Produce a JUnit report as well with --report-xunit-junit and add a few lines next to test-summary.py that list the three slowest tests in the JUnit file.
★★★ Change the integration test project to work with Testcontainers: if ConnectionStrings__Bookstore is defined, use it; otherwise start its own PostgreSQL container. Try both ways with act and compare the durations.
The API side of CI is ready. In the next post we apply the same discipline to the Angular app: reading the Node version from package.json with setup-node@v7, npm ci, ESLint, Vitest, build budgets and Playwright e2e tests that bring the API up together with PostgreSQL.