dotnet tools and dotnet new Templates: Local Tool Manifests, dnx, Packaging Your Own Tool and Template

Version note: The code in this post was built and run with .NET 10 (SDK 10.0.404); the Reporting examples were run with SDK 11.0.100 and the internal feed with BaGetter 1.6.5.

Guess First

Emre added this line to a nightly report script of Bookstore, the online bookstore used as the shared example across this site's series, and tried it on his machine:

dotnet bookstore-admin info

The script worked on his machine and ended like this on the CI agent:

Could not execute because the specified command or file was not found.
Possible reasons for this include:
  * You misspelled a built-in dotnet command.
  * You intended to execute a .NET program, but dotnet-bookstore-admin does not exist.
  * You intended to run a global tool, but a dotnet-prefixed executable with this name could not be found on the PATH.

Same command, same SDK. Where was the difference? Months ago Emre had installed the tool as a global tool; it lived in his user profile, not in the repo. In this post we tie .NET tools to the repo with a local tool manifest, run tools without installing them with dnx, turn bookstore-admin into a tool package and write a dotnet new template for new modules. At the end we look at file-based apps, which need no project at all.

Global Tools, Local Tools and the Manifest

A .NET tool is a special NuGet package that carries a console app. It can be installed in three places: in the user profile with -g (global), in a folder you choose with --tool-path, or in a manifest file (local). A global tool is a single version for everyone on that machine and does not come with the repo; that was Emre's problem. A local tool is written to dotnet-tools.json in the repo, its version is committed, and everyone uses the same version. Bookstore's root already has a manifest with EF Core's dotnet-ef tool:

{
  "version": 1,
  "isRoot": true,
  "tools": {
    "dotnet-ef": {
      "version": "10.0.15",
      "commands": [
        "dotnet-ef"
      ],
      "rollForward": false
    }
  }
}

dotnet tool restore

Tool 'dotnet-ef' (version '10.0.15') was restored. Available commands: dotnet-ef

A newer version of tool 'dotnet-ef' is available (version '11.0.0'). Consider updating your manifest file.

Restore was successful.

dotnet tool restore downloads every tool in the manifest into NuGet's global packages folder; nothing is installed, the package is just ready. After that, dotnet ef or dotnet tool run dotnet-ef runs the version from the manifest. The new-version notice came from SDK 10.0.404, but with rollForward: false nothing is upgraded unless the manifest changes. We don't want the EF Core 11 tool in a .NET 10 app, so we ignore the notice on purpose.

isRoot: true stops the CLI from going further up the folder tree while looking for a manifest. Without a manifest, a command like dotnet bookstore-admin first checks the built-in commands and then dotnet- prefixed files on the PATH; the error message in the opening describes exactly that order. The rule for CI is simple: the pipeline starts with dotnet tool restore, and no script assumes a global tool.

Packaging bookstore-admin as a Tool

The reporting team wants to use bookstore-admin in their own repo. Instead of copying the source code, we packaged the tool and put it on the internal feed we set up in post 4. A few properties in the project file are enough:

<PropertyGroup>
  <OutputType>Exe</OutputType>
  <AssemblyName>bookstore-admin</AssemblyName>
  <!-- Trim, single-file and AOT analyzers run on every build, not only on publish. -->
  <IsAotCompatible>true</IsAotCompatible>
  <!-- dotnet pack produces a .NET tool package; the command name defaults to AssemblyName. -->
  <PackAsTool>true</PackAsTool>
  <PackageId>Bookstore.Admin</PackageId>
  <Version>1.0.0</Version>
  <Description>Admin commands for Bookstore: info, ISBN check, price formatting.</Description>
</PropertyGroup>

dotnet pack tools/Bookstore.Admin

  Bookstore.Admin -> C:\src\Bookstore\artifacts\publish\Bookstore.Admin\release\
  Successfully created package 'C:\src\Bookstore\artifacts\package\release\Bookstore.Admin.1.0.0.nupkg'.

Note the publish line in the output: a tool package is really a packaged publish folder. Under tools/net10.0/any/ it holds the dlls, appsettings.json and a DotnetToolSettings.xml generated by the SDK:

<DotNetCliTool Version="1">
  <Commands>
    <Command Name="bookstore-admin" EntryPoint="bookstore-admin.dll" Runner="dotnet" />
  </Commands>
</DotNetCliTool>

We didn't write ToolCommandName; the command name came from AssemblyName. Runner="dotnet" and the any folder say the tool is framework-dependent and platform-agnostic, so the machine needs the .NET 10 runtime. Since .NET 10, tool packages can also be RID-specific, self-contained or Native AOT; all the publish options from the previous post apply to tools. Bookstore's tools didn't need that, and we didn't try it.

We pushed the package to the feed, created a manifest in the Reporting repo and installed the tool. Reporting's global.json selects SDK 11.0.100, and the feed's user name and password are in environment variables as in post 4:

cd samples/reporting
dotnet new tool-manifest
dotnet tool install Bookstore.Admin

The template "Dotnet local tool manifest file" was created successfully.

You can invoke the tool from this directory using the following commands: 'dotnet tool run bookstore-admin' or 'dotnet bookstore-admin'.
Tool 'bookstore.admin' (version '1.0.0') was successfully installed. Entry is added to the manifest file C:\src\Bookstore\samples\reporting\dotnet-tools.json.

We didn't write --local, because local is the default for dotnet tool install; unless you pass -g, it adds the tool to the manifest. The source mapping in Reporting's nuget.config routes Bookstore.* packages to the internal feed only, so the tool was never even looked up on nuget.org.

dotnet bookstore-admin isbn 978-0-441-17271-9
$env:BOOKSTORE_ENVIRONMENT = "Staging"; dotnet tool run bookstore-admin info

valid: 9780441172719
Runtime:     .NET 10.0.15
Environment: Staging
API:         http://localhost:5080

The API address comes from the appsettings.json inside the package. An installed tool's folder lives in the NuGet cache and is not edited by hand; every setting that changes per environment should come from an environment variable or the command line. bookstore-admin already reads the environment from an environment variable; reading the API address the same way is a job for the next version.

We also tried a habit left over from an old project: giving the package metadata through a hand-written .nuspec file. SDK 11 brought a new warning for this; we tried it in a copy of the repo with global.json moved to 11.0.100:

dotnet pack tools/Bookstore.Admin -p:NuspecFile=bookstore-admin.nuspec

C:\Program Files\dotnet\sdk\11.0.100\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.PackTool.targets(283,5): warning NETSDK1235: .NET Tools do not support using a custom .nuspec file, but the nuspec file 'bookstore-admin.nuspec' was provided. .NET Tools use a custom package layout and package identifier convention and must be customized through MSBuild logic only. Remove the NuspecFile property from this project to enable packing it as a .NET Tool. [C:\src\Bookstore\tools\Bookstore.Admin\Bookstore.Admin.csproj]
  Successfully created package 'C:\src\Bookstore\artifacts\package\release\Bookstore.Admin.1.0.0.nupkg'.

The package was created despite the warning. When we looked inside, there was no tools/ folder; instead it contained Bookstore.Admin.csproj, Program.cs and the publish profile. A package that doesn't work and carries the source code would have gone to the feed. More importantly, TreatWarningsAsErrors did not turn this warning into an error: that property affects compiler and NuGet warnings, not the NETSDK warnings that come from the SDK's MSBuild tasks. Those need MSBuildTreatWarningsAsErrors; with it on, the build ended with an error, but the package file had already been written. Tool package metadata is set with MSBuild properties; the details of publishing packages are in the Versioning and Release Management series.

dnx: Running Without Installing

dnx, which came with .NET 10, downloads and runs a tool without adding it to a manifest; it is a shortcut for dotnet tool exec. If there is a manifest nearby, it uses the version from it:

cd samples/reporting
dnx Bookstore.Admin isbn 978-0-441-17271-9

valid: 9780441172719

Running the same command at the repo root gave a different result:

bookstore.admin is not found in NuGet feeds https://api.nuget.org/v3/index.json.

The root nuget.config only knows nuget.org. Adding the package folder from the command line didn't work either:

dnx Bookstore.Admin --add-source artifacts/package/release info

The --add-source option cannot be combined with package source mapping. To use an additional source, update your NuGet configuration file's source mapping settings. Learn more: https://aka.ms/nuget-package-source-mapping

That is a good outcome. dnx downloads a package from the internet and runs the code inside it under your account, and source mapping decides which package name may come from which feed. If the command line could punch through that rule, it would open a door for the dependency confusion attack we covered in post 4. According to the .NET 11 release notes, dnx and dotnet tool exec no longer ask for an extra confirmation; the only thing that controls where a package comes from is your NuGet configuration. Before you type a package name into dnx, apply the same care you would when adding it as a PackageReference.

A dotnet new Template: bookstore-module

Every time a new module is added to Bookstore, the same skeleton is set up by hand: a library under src/, a static class that registers services and maps endpoints, a test project under tests/, and two lines in the solution. We turned this into a template. A template is made of real files and a .template.config/template.json:

templates/Bookstore.Templates/
  Bookstore.Templates.csproj
  content/bookstore-module/
    .template.config/template.json
    .template.config/dotnetcli.host.json
    src/Bookstore.ModuleName/Bookstore.ModuleName.csproj
    src/Bookstore.ModuleName/ModuleNameModule.cs
    tests/Bookstore.ModuleName.Tests/Bookstore.ModuleName.Tests.csproj
    tests/Bookstore.ModuleName.Tests/ModuleNameModuleTests.cs

{
  "$schema": "http://json.schemastore.org/template",
  "author": "Bookstore",
  "classifications": [ "Bookstore", "Library" ],
  "identity": "Bookstore.Module",
  "name": "Bookstore module",
  "shortName": "bookstore-module",
  "sourceName": "ModuleName",
  "preferNameDirectory": false,
  "tags": { "language": "C#", "type": "project" },
  "symbols": {
    "includeTests": {
      "type": "parameter",
      "datatype": "bool",
      "defaultValue": "true",
      "description": "Also create a test project under tests/."
    }
  },
  "sources": [
    {
      "modifiers": [
        { "condition": "(!includeTests)", "exclude": [ "tests/**" ] }
      ]
    }
  ],
  "primaryOutputs": [
    { "path": "src/Bookstore.ModuleName/Bookstore.ModuleName.csproj" },
    { "condition": "(includeTests)", "path": "tests/Bookstore.ModuleName.Tests/Bookstore.ModuleName.Tests.csproj" }
  ],
  "postActions": [
    {
      "description": "Add the projects to Bookstore.slnx",
      "actionId": "D396686C-DE0E-4DE6-906D-291CD29FC5DE",
      "continueOnError": true,
      "manualInstructions": [ { "text": "Run dotnet sln add for the new projects." } ]
    }
  ]
}

sourceName is the heart of the template: ModuleName in file names, folder names and file contents is replaced with the name given with -n. The template engine also replaces the lowercase form, so /api/modulename in the route becomes /api/shipping on its own. The includeTests parameter makes the tests/ folder conditional. The GUID in postActions is the fixed identity of the template engine's "add projects to the solution" action; it takes the projects to add from primaryOutputs. dotnetcli.host.json sets the parameter's command-line name to --include-tests.

The files inside the template are ordinary C# files that compile. The module skeleton:

using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Routing;
using Microsoft.Extensions.DependencyInjection;

namespace Bookstore.ModuleName;

public static class ModuleNameModule
{
    public static IServiceCollection AddModuleNameModule(this IServiceCollection services) => services;

    public static IEndpointRouteBuilder MapModuleNameEndpoints(this IEndpointRouteBuilder app)
    {
        RouteGroupBuilder group = app.MapGroup("/api/modulename").WithTags("ModuleName");
        group.MapGet("/status", () => TypedResults.Ok("ModuleName module is running"));
        return app;
    }
}

The generated project file only has a FrameworkReference; TargetFramework, Nullable, the analyzers and the artifacts layout come from the Directory.Build.props of post 3. Because the template doesn't repeat the repo's rules, it doesn't have to be updated when those rules change.

Packaging and Testing the Template

To ship the template to the team we put it in a NuGet package too. Our first attempt was this project file, without the last two properties:

<PropertyGroup>
  <PackageType>Template</PackageType>
  <PackageId>Bookstore.Templates</PackageId>
  <Version>1.0.0</Version>
  <Description>dotnet new templates for the Bookstore solution.</Description>
  <!-- The package carries only the content folder: no assembly, no compiled code. -->
  <IncludeBuildOutput>false</IncludeBuildOutput>
  <!-- A template package has no lib folder and no dependencies; otherwise NU5128. -->
  <SuppressDependenciesWhenPacking>true</SuppressDependenciesWhenPacking>
  <IncludeContentInPack>true</IncludeContentInPack>
  <ContentTargetFolders>content</ContentTargetFolders>
  <!-- Without this, NuGet drops dot-folders such as .template.config. -->
  <NoDefaultExcludes>true</NoDefaultExcludes>
</PropertyGroup>

<ItemGroup>
  <Content Include="content/**" Exclude="content/**/bin/**;content/**/obj/**" />
  <!-- The .cs files under content/ are template text, not part of this project. -->
  <Compile Remove="content/**" />
</ItemGroup>

Without Compile Remove, the first dotnet pack produced eleven compile errors:

C:\src\Bookstore\templates\Bookstore.Templates\content\bookstore-module\src\Bookstore.ModuleName\ModuleNameModule.cs(1,17): error CS0234: The type or namespace name 'AspNetCore' does not exist in the namespace 'Microsoft' (are you missing an assembly reference?) [C:\src\Bookstore\templates\Bookstore.Templates\Bookstore.Templates.csproj]

The SDK's **/*.cs glob took the template content for the package project's own source code. With that removed, the next error came from NuGet:

C:\Program Files\dotnet\sdk\10.0.404\NuGet.Build.Tasks.Pack.targets(226,5): error NU5128: Warning As Error: Some target frameworks declared in the dependencies group of the nuspec and the lib/ref folder do not have exact matches in the other location. Consult the list of actions below: [C:\src\Bookstore\templates\Bookstore.Templates\Bookstore.Templates.csproj]
C:\Program Files\dotnet\sdk\10.0.404\NuGet.Build.Tasks.Pack.targets(226,5): error NU5128: - Add lib or ref assemblies for the net10.0 target framework [C:\src\Bookstore\templates\Bookstore.Templates\Bookstore.Templates.csproj]

The package has no lib folder but its dependency list has a net10.0 group; the two disagree. The NuGet warning became an error because of TreatWarningsAsErrors. SuppressDependenciesWhenPacking removed the dependency group and the package was created. Both errors show that the repo's shared settings apply to the template project too; had we prepared the template package outside the repo, we would have seen neither.

dotnet new install artifacts/package/release/Bookstore.Templates.1.0.0.nupkg
dotnet new bookstore-module -n Shipping --dry-run
dotnet new bookstore-module -n Shipping

File actions would have been taken:
  Create: ./src/Bookstore.Shipping/Bookstore.Shipping.csproj
  Create: ./src/Bookstore.Shipping/ShippingModule.cs
  Create: ./tests/Bookstore.Shipping.Tests/Bookstore.Shipping.Tests.csproj
  Create: ./tests/Bookstore.Shipping.Tests/ShippingModuleTests.cs
...
The template "Bookstore module" was created successfully.

Processing post-creation actions...
Adding
    project(s): C:\src\Bookstore\src\Bookstore.Shipping\Bookstore.Shipping.csproj C:\src\Bookstore\tests\Bookstore.Shipping.Tests\Bookstore.Shipping.Tests.csproj
    to solution file: C:\src\Bookstore\Bookstore.slnx
    solution folder:
Project `src/Bookstore.Shipping/Bookstore.Shipping.csproj` added to the solution.
Project `tests/Bookstore.Shipping.Tests/Bookstore.Shipping.Tests.csproj` added to the solution.
Successfully added project(s) to a solution file.

--dry-run shows what would happen without writing to disk; it is the option you will use most while developing a template. In the real run, the post action added both projects to the /src/ and /tests/ folders in Bookstore.slnx. Testing the template means exactly this: building the generated code and running its tests.

dotnet build Bookstore.slnx
dotnet test --project tests/Bookstore.Shipping.Tests

Test run summary: Passed!
  total: 1
  failed: 0
  succeeded: 1
  skipped: 0

We did this in a copy of the repo; adding a Shipping module to Bookstore is not the topic of this post. We repeated the same steps with --include-tests false as well; only the library was created and a single project was added to the solution.

File-Based Apps: Scripts Without a Project

Not every tool needs a project and a package. The catalog team wants to check the ISBN lists that come from publishers, so we wrote a single-file program in the scripts/ folder:

#!/usr/bin/env dotnet
#:project ../src/Bookstore.Contracts/Bookstore.Contracts.csproj
#:include IsbnReport.cs

using Bookstore.Contracts;

// Reads one ISBN per line and reports the invalid ones; exit code 1 if any.
IsbnReport report = new();
foreach (string line in File.ReadLines(args[0]))
{
    report.Add(line, Isbn.TryParse(line, out _));
}

report.Print();
return report.InvalidCount == 0 ? 0 : 1;

dotnet run scripts/check-isbns.cs -- scripts/isbns.txt

invalid: 978-0-441-17271-0
1 invalid ISBN(s)

#:project connects the script to Bookstore.Contracts, so we don't rewrite the ISBN check. Inside a repo, a file-based app behaves like an ordinary project: Directory.Build.props, Directory.Packages.props, nuget.config and global.json apply to it as well. We saw this in an experiment: when we wrote DateTime.Now in a script, the banned API analyzer from post 3 kicked in:

C:\src\Bookstore\scripts\now.cs(1,42): error RS0030: The symbol 'DateTime.Now' is banned in this project: Inject TimeProvider instead (https://github.com/dotnet/roslyn/blob/main/src/RoslynAnalyzers/Microsoft.CodeAnalysis.BannedApiAnalyzers/BannedApiAnalyzers.Help.md)

If you want different rules for scripts, you can put a separate Directory.Build.props under scripts/. The build output also went to the repo's artifacts layout: dotnet build scripts/check-isbns.cs -getProperty:OutputPath returned artifacts/bin/debug/. PublishAot is on by default for file-based apps; if you publish the script, you get a Native AOT executable.

We moved the report class to a separate file with #:include. According to Microsoft's documentation this directive arrived in .NET 11 and in SDK 10.0.300; indeed, on 10.0.115 the script stopped with this error:

C:\src\Bookstore\scripts\check-isbns.cs(3): error: Unrecognized directive 'include'.

Bookstore's global.json requires the 10.0.4xx band, so the repo is fine, but seeing that a feature band difference can break even a script shows again why pinning the SDK in post 1 matters. The first version had no shebang line, and the build ended with this error:

C:\src\Bookstore\scripts\check-isbns.cs(2,1): error CA2266: File-based program entry point should start with '#!' (https://learn.microsoft.com/dotnet/fundamentals/code-analysis/quality-rules/ca2266)

In a file-based app with more than one file, CA2266 wants the entry file to start with #!, so tools can tell which file is the program and which one is an included part. The shebang line has no effect on Windows; dotnet run skips it. When the script grows beyond a few files, packages and tests, dotnet project convert scripts/check-isbns.cs turns it into an ordinary project.

In Production

Make dotnet tool restore the first step of the CI pipeline and keep tool versions pinned in the manifest. Upgrading a tool is done through a pull request, just like upgrading a package; a line in CI that says "install the latest version" means the build will change one day for no visible reason.

Tool packages are dependencies too. Keep who can push to the feed, where packages come from and source mapping as strict for tools as you do for library packages. Even a package run once with dnx executes code on the machine.

Templates are tested like code. A CI job that packs the template package, installs it into a copy of the repo, generates output with each combination of parameters and runs build and test shows that the template still matches the repo's rules. When a new analyzer rule is added to Directory.Build.props, the template output is usually the first thing to break.

Common Pitfalls

A CI script that depends on a global tool. A script that works on the developer's machine ends with "command not found" on a clean agent. Put tools in a local manifest and start the script with dotnet tool restore.

Publishing a template without testing it. Our template produced eleven compile errors and a NuGet error on its first pack; had these appeared on a teammate's machine after the package was published, it would have taken much longer to see that the problem was in the template.

Giving a tool package a custom .nuspec. The package is created but does not work as a tool, and SDK 11's NETSDK1235 warning does not become an error with TreatWarningsAsErrors.

Editing appsettings.json in an installed tool's folder. That folder is the NuGet cache; your change is lost on the next restore or version change. Pass settings that vary per environment through environment variables.

Running a file-based app on an older feature band. Directives like #:include depend on the SDK band; if no global.json covers the script's folder, the newest SDK on the machine is selected and the result changes from machine to machine.

A Small Experiment

★☆☆ Run dotnet tool restore at the repo root, then look at the tools in the manifest with dotnet tool list. Change the version of dotnet-ef in dotnet-tools.json, restore again and compare the output of dotnet ef --version.

★★☆ Add a bool parameter named --with-endpoints to the bookstore-module template; when it is false, the MapModuleNameEndpoints method should not be generated. Hint: C# files can use conditional regions in the form //#if (withEndpoints). Generate with both values and run build and test.

★★★ Make bookstore-admin read its API address from the BOOKSTORE_API_BASE_URL environment variable, bump the version to 1.1.0, push the package to the feed and update the Reporting manifest with dotnet tool update Bookstore.Admin. Judge the manifest change by how it would look in a pull request.

In this post we packaged tools and templates, and each time we touched targets the SDK runs behind the scenes: pack calling publish, the glob compiling template content, a warning that does or does not become an error. In the next post we go down to that layer itself: MSBuild's properties, items, targets and tasks, writing our own target, and using a binlog to see what happens inside a build.