Mert Özen Further With Every Line
DevOps Web Servers and Hosting — #08

Running a .NET App as a systemd Service on Linux

Mert Özen Nov 8, 2026 12 min 4 views
Running a .NET App as a systemd Service on Linux

Run Kestrel under systemd as a separate user with Type=notify and AddSystemd, KillSignal=SIGTERM and hardening settings; KillSignal=SIGINT cuts open requests and systemd still reports success. We measure publish option sizes, the runtime coming from the distribution repository on Ubuntu 24.04, journald output, restart after a crash, the exposure score dropping from 9.2 to 3.8, and 0.2-second deploys and rollbacks with a releases/current symlink.

Version note: The examples in this post were run with the .NET 10 SDK (Software Development Kit) 10.0.402, aspnetcore-runtime-10.0 10.0.14 from the Ubuntu 24.04 package repository, Microsoft.Extensions.Hosting.Systemd 10.0.14 and systemd 255, in an Ubuntu 24.04 container running systemd as PID (process ID) 1 on Docker 29.

Guess First

Bookstore.Api runs as a systemd service on a Linux VM (virtual machine). An old unit file someone found on the internet says KillSignal=SIGINT; someone else wants to change it to KillSignal=SIGTERM. As a test, a report request that takes 5 seconds is started and one second later systemctl stop bookstore-api is run.

Guess: does the report request complete with both settings? How long does systemctl stop take? And if there's no difference between them, why does Microsoft's documentation explicitly tell you to avoid SIGINT (the interrupt signal, what Ctrl+C sends in a terminal)?

Publish Options: Framework-Dependent, Self-Contained, Single-File

The options of dotnet publish decide what you copy to the server. Four outputs for the same Bookstore.Api:

dotnet publish -c Release -o out/pub-fdd
dotnet publish -c Release -r linux-arm64 --self-contained false -o out/pub-fdd-rid
dotnet publish -c Release -r linux-arm64 --self-contained true -o out/pub-scd
dotnet publish -c Release -r linux-arm64 --self-contained true -p:PublishSingleFile=true -o out/pub-single
OutputSizeFilesNeeded on the server
Framework-dependent, no RID (Runtime Identifier)232K10ASP.NET Core runtime; runs with dotnet Bookstore.Api.dll
Framework-dependent, linux-arm64180K9Runtime; plus a ./Bookstore.Api executable
Self-contained114M337Nothing; the runtime is inside the output
Self-contained, single-file106M5Nothing; a single 111 MB executable

Framework-dependent output is small and uses the runtime on the server; the runtime's security patches arrive with apt upgrade and apply to every app. Self-contained output carries the runtime inside: you don't need to install anything on the server, and apps on different .NET versions run side by side, but the runtime's patches arrive only when you republish the app. The framework-dependent output with a RID has no web.config because the RID isn't Windows; the output without a RID also contains the web.config generated for IIS (Internet Information Services), harmless on Linux but unnecessary. In this post I use the framework-dependent output without a RID.

Installing the Runtime: Ubuntu's Repository or Microsoft's?

On Ubuntu 24.04, .NET 10 is directly in the distribution's own repository:

$ apt-cache policy aspnetcore-runtime-10.0 aspnetcore-runtime-8.0 | grep -E "^[a-z]|Candidate"
aspnetcore-runtime-10.0:
  Candidate: 10.0.14-0ubuntu1~24.04.1
aspnetcore-runtime-8.0:
  Candidate: 8.0.33-0ubuntu1~24.04.1
$ sudo apt-get install -y aspnetcore-runtime-10.0
$ dotnet --list-runtimes
Microsoft.AspNetCore.App 10.0.14 [/usr/lib/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 10.0.14 [/usr/lib/dotnet/shared/Microsoft.NETCore.App]

According to the table on the "Install .NET on Ubuntu" page on Microsoft Learn, for Ubuntu 24.04 .NET 10 and 8 are in the distribution's own repository and 9.0 in Ubuntu's backports repository; Microsoft's package repository (packages.microsoft.com) has no .NET for this release. The "add the Microsoft repository first" step in older guides is unnecessary on newer Ubuntu releases, and adding both repositories leads to the package mix-ups the documentation separately warns about. The installation goes to /usr/lib/dotnet and /usr/bin/dotnet links there; know that this is the path in your unit file, older examples that say /usr/local/bin/dotnet won't work with this installation.

The Unit File Line by Line

A separate system user without login rights for the app, release folders, and a writable folder for Data Protection keys:

sudo useradd --system --no-create-home --shell /usr/sbin/nologin bookstore
sudo mkdir -p /var/www/bookstore/releases/2026-11-08-1 /var/lib/bookstore/keys /etc/bookstore
sudo cp -r out/pub-fdd/. /var/www/bookstore/releases/2026-11-08-1/
sudo ln -sfn /var/www/bookstore/releases/2026-11-08-1 /var/www/bookstore/current
sudo chown -R bookstore:bookstore /var/lib/bookstore

/etc/systemd/system/bookstore-api.service:

[Unit]
Description=Bookstore API (ASP.NET Core on Kestrel)
After=network.target

[Service]
Type=notify
User=bookstore
Group=bookstore
WorkingDirectory=/var/www/bookstore/current
ExecStart=/usr/bin/dotnet /var/www/bookstore/current/Bookstore.Api.dll
Restart=always
RestartSec=10
KillSignal=SIGTERM
TimeoutStopSec=30
SyslogIdentifier=bookstore-api
Environment=ASPNETCORE_ENVIRONMENT=Production
Environment=ASPNETCORE_URLS=http://127.0.0.1:5000
Environment=DOTNET_NOLOGO=true
Environment=DataProtection__KeysDirectory=/var/lib/bookstore/keys
EnvironmentFile=-/etc/bookstore/api.env

[Install]
WantedBy=multi-user.target
  • Type=notify: systemd considers the service running not when the process starts but when the app says "I'm ready". That needs AddSystemd() in the app; see below.
  • User and Group: so it doesn't run as root. Without this line the service runs as root.
  • WorkingDirectory and ExecStart: through the current symlink; on deploy only the link changes.
  • Restart=always and RestartSec=10: whatever ends the process, start it again after 10 seconds.
  • KillSignal and TimeoutStopSec: which signal to send when stopping and how long to wait before SIGKILL (the signal that can't be caught and kills the process at once). Microsoft's documentation says the default is 90 seconds; the topic of the next section.
  • Environment: the binding rules from the second post apply here too; Kestrel listens only on loopback, for the Nginx in front. Use __ instead of : for nested configuration keys.
  • EnvironmentFile=-/etc/bookstore/api.env: secrets and server-specific values live outside the unit file, in a file only root can read. The leading - means don't fail if the file is missing.

AddSystemd: Type=notify and journald

The Microsoft.Extensions.Hosting.Systemd package and one line:

builder.Services.AddSystemd();

If the app isn't running under systemd, this line does nothing; the same code runs unchanged in Docker and on a developer machine. Under systemd three things change: once the host is ready a READY=1 notification goes to systemd, a lifetime suited to systemd is used for the stop signal, and the console log turns into a format journald understands. Let's start it and look:

$ sudo systemctl daemon-reload
$ sudo systemctl enable --now bookstore-api
Created symlink /etc/systemd/system/multi-user.target.wants/bookstore-api.service → /etc/systemd/system/bookstore-api.service.
$ journalctl -u bookstore-api --no-pager
Nov 08 04:32:12 bookstore-vm systemd[1]: Starting bookstore-api.service - Bookstore API (ASP.NET Core on Kestrel)...
Nov 08 04:32:12 bookstore-vm bookstore-api[396]: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[35] No XML encryptor configured. Key {ea7719a5-ce79-4c02-aa6b-98753116d198} may be persisted to storage in unencrypted form.
Nov 08 04:32:12 bookstore-vm bookstore-api[396]: Microsoft.Hosting.Lifetime[14] Now listening on: http://127.0.0.1:5000
Nov 08 04:32:12 bookstore-vm bookstore-api[396]: Microsoft.Hosting.Lifetime[0] Application started. Hosting environment: Production; Content root path: /var/www/bookstore/releases/2026-11-08-1
Nov 08 04:32:12 bookstore-vm systemd[1]: Started bookstore-api.service - Bookstore API (ASP.NET Core on Kestrel).

Three details. The "Started" line comes after the app's "Application started" line: thanks to Type=notify, systemd waited until Kestrel opened the port. That means when systemctl start or restart returns, the app really accepts requests; that's why the smoke test in the deploy script can run right away. Second, the log lines are single lines without colour codes; each line's journald priority comes from its log level, so journalctl -p warning filters warnings only:

$ journalctl -u bookstore-api -p warning --no-pager
Nov 08 04:32:12 bookstore-vm bookstore-api[396]: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[35] No XML encryptor configured. Key {ea7719a5-ce79-4c02-aa6b-98753116d198} may be persisted to storage in unencrypted form.

Third, "Content root path" shows the real release folder, not the symlink; the easiest way to read from the log which version is running. Is the app really running as the bookstore user:

$ ps -o user,pid,cmd -C dotnet
USER         PID CMD
booksto+     396 /usr/bin/dotnet /var/www/bookstore/current/Bookstore.Api.dll

Graceful Shutdown and Restart after a Crash

Bookstore.Api writes two log lines while the host stops (the ApplicationStopping and ApplicationStopped callbacks). systemctl stop:

Nov 08 04:32:22 bookstore-vm systemd[1]: Stopping bookstore-api.service - Bookstore API (ASP.NET Core on Kestrel)...
Nov 08 04:32:22 bookstore-vm bookstore-api[396]: Microsoft.Hosting.Lifetime[0] Application is shutting down...
Nov 08 04:32:22 bookstore-vm bookstore-api[396]: Bookstore.Api[0] Stopping: draining in-flight requests on vm1
Nov 08 04:32:22 bookstore-vm bookstore-api[396]: Bookstore.Api[0] Stopped cleanly on vm1
Nov 08 04:32:22 bookstore-vm systemd[1]: bookstore-api.service: Deactivated successfully.
Nov 08 04:32:22 bookstore-vm systemd[1]: Stopped bookstore-api.service - Bookstore API (ASP.NET Core on Kestrel).

The same with KillSignal=SIGINT:

Nov 08 04:32:24 bookstore-vm systemd[1]: Stopping bookstore-api.service - Bookstore API (ASP.NET Core on Kestrel)...
Nov 08 04:32:24 bookstore-vm systemd[1]: bookstore-api.service: Deactivated successfully.
Nov 08 04:32:24 bookstore-vm systemd[1]: Stopped bookstore-api.service - Bookstore API (ASP.NET Core on Kestrel).
$ systemctl show bookstore-api -p Result -p ExecMainStatus -p ExecMainCode
Result=success
ExecMainCode=2
ExecMainStatus=2

Not a single line from the app. ExecMainCode=2 means "killed by a signal" and ExecMainStatus=2 is the signal number: SIGINT. The app ended without running any shutdown code, yet systemd reports it as Result=success, because it counts a process that ended with the signal it sent itself as successful. The answer to the guess, with the 5-second report request:

=== KillSignal=SIGTERM
in-flight request: 200 after 5.034589s
curl exit=0
systemctl stop took 4.045s
=== KillSignal=SIGINT
in-flight request: 000 after 1.006153s
curl exit=52
systemctl stop took 0.010s

With SIGTERM (the terminate signal, "get ready to shut down") Kestrel stopped accepting new connections, waited for the open request to finish (the stop took 4 seconds) and the client got a 200. With SIGINT the process went away instantly and the client got an empty reply. The reason is the lifetime AddSystemd() uses: under systemd it listens only for SIGTERM as the graceful shutdown signal; the console lifetime that catches the SIGINT coming from Ctrl+C in a terminal isn't in play. Microsoft's Nginx documentation says so explicitly: .NET's lifetime integrations listen for SIGTERM for graceful shutdown, avoid KillSignal=SIGINT. SIGTERM is already systemd's default; writing the line only documents the intent. TimeoutStopSec should be long enough for long requests and background jobs; when it runs out systemd sends SIGKILL. The shutdown timeout on the app side and how BackgroundService instances stop meanwhile are in the "IHostedService and BackgroundService" post of the Background Jobs and Scheduled Jobs series.

If the process crashes, Restart=always kicks in. I simulated it with kill -9:

Nov 08 04:33:04 bookstore-vm systemd[1]: bookstore-api.service: Main process exited, code=killed, status=9/KILL
Nov 08 04:33:04 bookstore-vm systemd[1]: bookstore-api.service: Failed with result 'signal'.
Nov 08 04:33:14 bookstore-vm systemd[1]: bookstore-api.service: Scheduled restart job, restart counter is at 1.
Nov 08 04:33:14 bookstore-vm systemd[1]: Starting bookstore-api.service - Bookstore API (ASP.NET Core on Kestrel)...
Nov 08 04:33:15 bookstore-vm bookstore-api[770]: Microsoft.Hosting.Lifetime[14] Now listening on: http://127.0.0.1:5000
Nov 08 04:33:15 bookstore-vm bookstore-api[770]: Microsoft.Hosting.Lifetime[0] Application started. Hosting environment: Production; Content root path: /var/www/bookstore/releases/2026-11-08-1
Nov 08 04:33:15 bookstore-vm systemd[1]: Started bookstore-api.service - Bookstore API (ASP.NET Core on Kestrel).
$ systemctl show bookstore-api -p NRestarts
NRestarts=1

The 10-second wait keeps a constantly crashing app from burning the CPU (Central Processing Unit). Wire the NRestarts value into monitoring: a service that quietly restarts is a bug nobody noticed.

Hardening: systemd-analyze security

Running as a separate user is the first step. systemd can narrow the process's access to the file system, the kernel and the network much further, and it scores how exposed it is itself:

$ systemd-analyze security bookstore-api --no-pager | tail -1
→ Overall exposure level for bookstore-api.service: 9.2 UNSAFE :-{

I added the settings with a drop-in file without touching the unit file (/etc/systemd/system/bookstore-api.service.d/hardening.conf):

[Service]
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
PrivateTmp=true
PrivateDevices=true
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
RestrictNamespaces=true
LockPersonality=true
CapabilityBoundingSet=
ReadWritePaths=/var/lib/bookstore
$ sudo systemctl daemon-reload && sudo systemctl restart bookstore-api
$ systemctl is-active bookstore-api
active
$ curl -s -o /dev/null -w "%{http_code}\n" http://127.0.0.1:5000/api/books
200
$ curl -s "http://127.0.0.1:5000/api/dp/protect?text=cart-42" | cut -c1-20
CfDJ8KUZd-p5zgJMqmuY
$ systemd-analyze security bookstore-api --no-pager | tail -1
→ Overall exposure level for bookstore-api.service: 3.8 OK :-)

ProtectSystem=strict makes the whole file system read-only; the only place the app needs to write is the Data Protection keys, and that's opened with ReadWritePaths. NoNewPrivileges stops the process and its children from gaining privileges, PrivateTmp stops it from seeing other services' /tmp, an empty CapabilityBoundingSet blocks all Linux capabilities. Since the app listens on port 5000 (not below 1024), it needs no capability at all. After each setting, check whether there's somewhere the app tries to write: a log file, an upload folder, a cache. A "Read-only file system" error means a missing ReadWritePaths. The same unit structure applies to Worker Services; the "Worker Service and In-Process Job Queue" post of the Background Jobs series continues from there.

Copying the new version over the running folder creates two problems: during the copy the app can run with half old and half new files, and to go back the old files no longer exist. Each version goes into its own folder, and the current symlink only points at the running one:

#!/bin/bash
# deploy.sh <release-name> <publish-dir>: copy, switch symlink, restart, smoke test
set -euo pipefail
release=$1; src=$2
base=/var/www/bookstore
mkdir -p "$base/releases/$release"
cp -r "$src/." "$base/releases/$release/"
ln -sfn "$base/releases/$release" "$base/current.tmp"
mv -T "$base/current.tmp" "$base/current"
systemctl restart bookstore-api
curl -sf -o /dev/null http://127.0.0.1:5000/health && echo "deployed $release: healthy"

Instead of changing the symlink directly with ln -sfn, creating a temporary link and moving it into place with mv -T turns the change into a single atomic rename; there's never a moment without current.

$ time deploy.sh 2026-11-08-2 /mnt/out/pub-fdd
deployed 2026-11-08-2: healthy
took 0.200s
$ readlink /var/www/bookstore/current
/var/www/bookstore/releases/2026-11-08-2

# rollback: point the link at the old folder, restart
$ time (ln -sfn /var/www/bookstore/releases/2026-11-08-1 /var/www/bookstore/current.tmp && mv -T /var/www/bookstore/current.tmp /var/www/bookstore/current && systemctl restart bookstore-api)
took 0.168s
$ journalctl -u bookstore-api --no-pager | grep "Content root" | tail -1
Nov 08 04:33:46 bookstore-vm bookstore-api[952]: Microsoft.Hosting.Lifetime[0] Application started. Hosting environment: Production; Content root path: /var/www/bookstore/releases/2026-11-08-1

Rollback takes 0.17 seconds; the old version is waiting on disk. Thanks to Type=notify, systemctl restart returned only once the app was ready, and the smoke test passed immediately. But the restart itself is a short outage: requests arriving in between get a 502 from Nginx. For deploys without downtime you need two instances and a drain in Nginx one at a time, or blue-green; those are the topic of the upcoming Deployment Strategies series. Also add a cleanup step to the script that keeps a few release folders and deletes the rest.

In Production

Don't write secrets in the unit file. systemctl show bookstore-api -p Environment shows the unit's Environment lines to everyone; values given with EnvironmentFile are protected by the file's permissions (chmod 600, owned by root). If a value contains spaces or special characters, use systemd-escape as Microsoft recommends.

Know where the logs go. journald rotates with a disk quota by default and deletes old entries; for long retention and search you need to ship logs to a central place. The "Serilog Sinks" post of the Logging in .NET series covers that path; journald itself is only a local buffer.

Restart the app when you update the runtime. The last note in Microsoft's documentation is exactly this: when the shared framework is updated, the running process keeps using the old files. systemctl restart bookstore-api after apt upgrade should be part of the maintenance procedure.

Common Pitfalls

Misspelling the name of appsettings.Production.json. Linux's file system is case-sensitive:

appsettings.production.json -> "instance":"bookstore-vm"
appsettings.Production.json -> "instance":"from-appsettings"

The lowercase file is read on Windows and silently ignored on Linux. The app raises no error; it runs with the defaults instead of the production settings.

Running the service as root. Without a User= line systemd starts the service as root; a hole in the app means access to the whole server.

KillSignal=SIGINT. This line, left over from old guides, cuts open requests on every deploy and systemd reports it as "success".

Keeping the Unix socket in /tmp or a hand-made folder. When the process crashes the socket file stays behind and the next start fails with "address already in use"; I ran into this in the container experiment in the second post of the series. With RuntimeDirectory=bookstore systemd creates /run/bookstore owned by the service user when the service starts and deletes it when it stops; in this container, when I killed the process with SIGKILL the folder was removed and the restart went fine.

Copying the new version over the running folder. There's no way back and the app can run with mixed files during the copy.

A Small Experiment

★☆☆ Remove the User= and Group= lines from the unit file and look at the systemd-analyze security score and the ps output. Then replace the lines with DynamicUser=yes; use StateDirectory=bookstore for the Data Protection keys and find where the keys are written.

★★☆ Set TimeoutStopSec=3 and run systemctl stop during a 10-second report request. What do you see in the journal? Also change HostOptions.ShutdownTimeout on the app side and explain how the two timeouts relate.

★★★ Set up the same unit with the self-contained single-file output (ExecStart=/var/www/bookstore/current/Bookstore.Api), remove aspnetcore-runtime-10.0 from the server and verify it runs with the hardening settings. Is there a folder the single-file executable extracts anything to at startup, and does ProtectSystem=strict allow it?

The Linux side is done: Nginx in front, Kestrel under systemd, deploy and rollback in a few commands. In the next post we move the same app to Windows Server: IIS, the Hosting Bundle, the ASP.NET Core Module, in-process and out-of-process hosting, and what errors such as 500.30 are telling you.