Getting an Express API Production-Ready: Configuration, Structured Logging with pino, Health Checks, Graceful Shutdown and Cluster

Version note: The examples in this post were run with Node.js 26.13.1 (LTS), Express 5.3.0, pino 10.4.0, zod 4.6.7, mssql 12.7.14, TypeScript 7.0.2 and Docker 29.

Guess First

In Bookstore, the online bookstore used as the shared example across this site's series, the API connected to SQL Server at the end of post 13. Gül from customer service is downloading the 98 MB order export; at that very moment a new version is deployed and the orchestrator sends SIGTERM to the old process. This is how server.ts ended in post 13:

const server = app.listen(port, host, (err?: Error) => {
    if (err) {
        console.error(`cannot listen on ${host}:${port}: ${err.message}`);
        process.exit(1);
    }
    console.log(`Bookstore API listening on http://${host}:${port} (${app.get("env")})`);
});

There is no code for signals at all. One second after the download starts we send the process a SIGTERM (curl is limited to 20 MB per second, so the full download takes about 5 seconds). What happens to Gül's file?

curl -s -o /dev/null --limit-rate 20M -w "export: %{http_code} %{size_download} bytes %{time_total}s curl-exit=%{exitcode}\n" http://127.0.0.1:3001/api/orders/export
kill -TERM <api-pid>

export: 200 21842287 bytes 1.040896s curl-exit=18

Status 200, but 22 MB arrived instead of 98 MB; curl's error 18 means "the transfer was cut short". Without a SIGTERM listener Node applies the default behaviour and the process dies on the spot: open responses, unwritten log lines, SQL connections, all left where they were. The status line has already gone out, so the client does not even see a 500; it holds half a CSV. In this post we get the API ready for production: configuration validated against a schema, structured logging with pino, liveness and readiness endpoints, graceful shutdown, PID 1 in a container, and the choice between cluster and replicas. I run the shutdown examples on port 3001.

Configuration: One Object Validated Against a Schema

Up to post 13, process.env.PORT, process.env.HOST and process.env.DB_CONNECTION_STRING were read in different places in the code. In ASP.NET Core the counterpart is appsettings.json + environment variables + the Options pattern: configuration is read once, bound to a typed object and validated at startup with ValidateOnStart. In Node we do the same with zod, which we used in post 12:

import { z } from "zod";

// Everything the process reads from its environment, validated once at startup (appsettings + Options pattern)
const environmentSchema = z.object({
    NODE_ENV: z.enum(["development", "production", "test"]).default("development"),
    HOST: z.string().min(1).default("127.0.0.1"),
    PORT: z.coerce.number().int().min(1).max(65535).default(3000),
    LOG_LEVEL: z.enum(["fatal", "error", "warn", "info", "debug", "trace"]).default("info"),
    DB_CONNECTION_STRING: z.string().min(1),
    SHUTDOWN_DRAIN_MS: z.coerce.number().int().min(0).max(60_000).default(0),
    SHUTDOWN_TIMEOUT_MS: z.coerce.number().int().min(1_000).max(120_000).default(10_000),
});

// Reads process.env (or a test's own object); every problem is reported at once, not one per restart
export function loadConfig(environment: Record<string, string | undefined> = process.env): AppConfig {
    const result = environmentSchema.safeParse(environment);
    if (!result.success) {
        throw new ConfigError(z.prettifyError(result.error));
    }
    const e = result.data;
    return Object.freeze({
        env: e.NODE_ENV,
        host: e.HOST,
        port: e.PORT,
        logLevel: e.LOG_LEVEL,
        db: Object.freeze({ connectionString: e.DB_CONNECTION_STRING }),
        shutdown: Object.freeze({ drainMs: e.SHUTDOWN_DRAIN_MS, timeoutMs: e.SHUTDOWN_TIMEOUT_MS }),
    });
}

Environment variables are always strings; z.coerce.number() turns "3001" into a number and rejects "abc". The result is a frozen (Object.freeze) object of type AppConfig; the rest of the code never touches process.env. Let's start with three bad values:

PORT=abc LOG_LEVEL=verbose node src/server.ts; echo "exit $?"

invalid configuration:
✖ Invalid input: expected number, received NaN
  → at PORT
✖ Invalid option: expected one of "fatal"|"error"|"warn"|"info"|"debug"|"trace"
  → at LOG_LEVEL
✖ Invalid input: expected string, received undefined
  → at DB_CONNECTION_STRING
exit 1

Three problems at once, and the process exits with code 1 before starting anything. That is the cheapest kind of failure in a deployment: the new pod never becomes ready and the old version keeps running. Because loadConfig takes an object as a parameter, tests can call it without touching process.env.

We introduced --env-file in post 02; it is for development only (npm run dev uses it). npm start reads no file: in production the platform provides the values (the container environment, a Kubernetes Secret, a secret store). Two more values belong to configuration: HOST must be 127.0.0.1 in development and 0.0.0.0 in a container, otherwise nobody outside the container can connect (post 10). We also read NODE_ENV here and hand it to Express with app.set("env", config.env); as we saw in post 11, Express's error page uses it to decide whether to show a stack trace, and it made no measurable difference to speed.

Structured Logging with pino

Since post 07 we had our own small JSON logger. For production we replace it with pino: one of the most widely used structured loggers in the Node ecosystem, it writes JSON lines cheaply and comes with log levels, child loggers and redaction. Think of it as Node's counterpart to Serilog.

import { pino } from "pino";
import { currentRequestContext } from "../context/requestContext.ts";

// One JSON line per entry on stdout; collecting and shipping logs is the platform's job
export const logger = pino({
    level: "info",
    base: { service: "bookstore-api" },
    timestamp: pino.stdTimeFunctions.isoTime,
    // "level":"info" instead of "level":30, as most log platforms expect
    formatters: { level: (label) => ({ level: label }) },
    // Added to every entry: the request id travels with AsyncLocalStorage, not as a parameter
    mixin() {
        const context = currentRequestContext();
        return context ? { requestId: context.requestId } : {};
    },
    // Secrets never reach the log, even when someone logs the whole object
    redact: {
        paths: ["config.db.connectionString", "*.password", "req.headers.authorization", "req.headers.cookie"],
        censor: "[redacted]",
    },
});

Calls switch to pino's order: fields first, then the message (logger.info({ orderId: order.id, items: items.length }, "order placed")). mixin reads the request id from the AsyncLocalStorage we set up in post 07 and adds it to every line; the id is not passed around as a parameter anywhere in the code. Because we provide base ourselves, the pid and hostname fields pino adds by default are gone too; in a container the hostname is the pod name and is useful, and then it goes into base. Let's place an order and look at the log:

{"level":"info","time":"2026-11-14T19:53:16.404Z","service":"bookstore-api","config":{"env":"development","host":"127.0.0.1","port":3000,"logLevel":"info","db":{"connectionString":"[redacted]"},"shutdown":{"drainMs":0,"timeoutMs":10000}},"msg":"configuration loaded"}
{"level":"info","time":"2026-11-14T19:53:16.537Z","service":"bookstore-api","url":"http://127.0.0.1:3000","env":"development","pid":92223,"msg":"Bookstore API listening"}
{"level":"info","time":"2026-11-14T19:53:18.404Z","service":"bookstore-api","requestId":"req-14a","orderId":"ORD-501005","items":1,"msg":"order placed"}
{"level":"info","time":"2026-11-14T19:53:18.408Z","service":"bookstore-api","requestId":"req-14a","method":"POST","path":"/api/orders","status":201,"durationMs":162,"msg":"request finished"}
{"level":"info","time":"2026-11-14T19:53:18.435Z","service":"bookstore-api","requestId":"req-14a","to":"ada@example.com","subject":"Order ORD-501005 received","msg":"e-mail sent"}

Three things to notice. First, at startup we log the whole configuration and the connection string shows up as [redacted]; without redaction the SQL Server password would end up on the log platform and from there with everyone who can read logs. Second, the "e-mail sent" line was written 27 ms after the response went out, yet it still carries req-14a: the async work in the event listener runs in the same request context. Third, the order number continues after the test data from post 13 (ORD-501005). Errors are written with logger.error({ err }, "unhandled error"); pino expands the Error object under the err key into type, message and stack.

Reading JSON during development is tiring; pino-pretty is installed as a devDependency and makes the output readable:

PORT=3001 node src/server.ts | npx pino-pretty

[22:55:45.849] INFO: request finished
    service: "bookstore-api"
    requestId: "req-14p"
    method: "GET"
    path: "/api/books/9780441172719"
    status: 200
    durationMs: 143

pino-pretty converts the time to local time; time in the JSON is UTC. Pretty printing is not used in production: the log collector expects JSON. LOG_LEVEL sets the level; once configuration is loaded, server.ts runs logger.level = config.logLevel. If you want a middleware that logs every HTTP request automatically, there is pino-http; we kept the "request finished" line of the requestContext middleware from post 07. Linking these logs to traces with OpenTelemetry and shipping them to a platform is the subject of the Observability series.

Health Checks: Liveness and Readiness

An orchestrator asks two different questions about a process: "Is this process alive, or should I restart it?" (liveness) and "Can I send traffic to it?" (readiness). The probes in the Kubernetes series call exactly these two endpoints. /health, which we have had since post 02, stays as liveness; we add readiness next to it:

// GET /health/ready (readiness): should traffic be sent here? 503 takes the instance out of the load balancer
export function readiness(check: ReadinessCheck): RequestHandler {
    return async (req, res) => {
        const report = await check();
        res.status(report.status === "ready" ? 200 : 503).json(report);
    };
}

// Ready = not shutting down and SQL Server answers within a second
export function createReadinessCheck(pool: sql.ConnectionPool, lifecycle: Lifecycle): ReadinessCheck {
    return async (): Promise<ReadinessReport> => {
        if (lifecycle.shuttingDown) {
            return { status: "not-ready", checks: { shutdown: "in progress" } };
        }
        try {
            const request = pool.request();
            const timer = setTimeout(() => request.cancel(), 1_000);
            await request.query("SELECT 1 AS ok").finally(() => clearTimeout(timer));
            return { status: "ready", checks: { sql: "ok" } };
        } catch (err) {
            return { status: "not-ready", checks: { sql: (err as Error).message } };
        }
    };
}

Liveness deliberately looks at no dependency. If liveness failed because SQL Server slowed down, the orchestrator would restart every API pod at once and make the problem worse. A database problem pulls readiness to 503; the instance stays alive but gets no traffic. The readiness check itself is bounded: if SELECT 1 does not return within a second, the request is cancelled. AppServices, which goes into createApp, gained readiness; in post 16 the tests will provide it as a fake function.

Graceful Shutdown

In ASP.NET Core the host catches SIGTERM itself, tells Kestrel to stop accepting connections, waits up to ShutdownTimeout for open requests to finish and stops the IHostedServices in order. In Node we write that sequence ourselves:

// SIGTERM (orchestrators, docker stop) and SIGINT (Ctrl+C) start the same graceful shutdown
export function registerGracefulShutdown({ server, pool, lifecycle, config }: ShutdownTargets): void {
    const shutdown = async (signal: NodeJS.Signals): Promise<void> => {
        if (lifecycle.shuttingDown) {
            return;
        }
        lifecycle.shuttingDown = true;
        const started = performance.now();
        logger.info({ signal }, "shutdown started: readiness now reports 503");

        // 1) Give the load balancer time to notice the 503 and stop sending traffic
        await delay(config.drainMs);

        // 2) No new connections; idle keep-alive connections close now, busy ones after their response
        const closed = new Promise<void>((resolve) => server.close(() => resolve()));
        const forced = delay(config.timeoutMs, "timeout" as const, { ref: false });
        if ((await Promise.race([closed.then(() => "closed" as const), forced])) === "timeout") {
            logger.warn({ timeoutMs: config.timeoutMs }, "requests still running at the deadline, closing their connections");
            server.closeAllConnections();
            await closed;
        }

        // 3) Only now close the database pool: no request can need it any more
        await pool.close();
        logger.info({ durationMs: Math.round(performance.now() - started) }, "shutdown complete");

        // Nothing should keep the event loop alive now; if something does, do not hang forever
        setTimeout(() => {
            logger.error("event loop still busy after shutdown, forcing exit");
            process.exit(1);
        }, 2_000).unref();
    };

    process.once("SIGTERM", shutdown);
    process.once("SIGINT", shutdown);
}

The order matters. Since Node 19, server.close() also closes idle keep-alive connections; connections that are handling a request close when their response ends, and the callback runs once all of them are closed. Had we closed the pool first, the export in progress would have lost its SQL connection. So that an unfinished request does not wait forever, closeAllConnections() kicks in when SHUTDOWN_TIMEOUT_MS runs out; the Node documentation recommends calling it after close(). The unref()'d timer at the end does not keep the process alive by itself; it only fires if something we forgot to close keeps the event loop alive for 2 more seconds. Normally it is not process.exit() but the emptied event loop that ends the process, and pino flushes its buffer along the way. Let's repeat the experiment from Guess First:

new request after SIGTERM: http=000 curl-exit=7
export: 200 97734499 bytes 4.655518s curl-exit=0

{"level":"info","time":"2026-11-14T19:53:58.308Z","service":"bookstore-api","signal":"SIGTERM","msg":"shutdown started: readiness now reports 503"}
{"level":"info","time":"2026-11-14T19:54:01.807Z","service":"bookstore-api","requestId":"req-14-export","method":"GET","path":"/api/orders/export","status":200,"durationMs":4573,"msg":"request finished"}
{"level":"info","time":"2026-11-14T19:54:01.807Z","service":"bookstore-api","requestId":"req-14-export","gzip":false,"durationMs":4573,"msg":"order export finished"}
{"level":"info","time":"2026-11-14T19:54:01.889Z","service":"bookstore-api","durationMs":3582,"msg":"shutdown complete"}

Gül's file arrived complete (97,734,499 bytes; slightly bigger than in post 13 because orders were added in between). A new connection opened 300 ms after SIGTERM was refused (curl's error 7: could not connect), and once the export finished the process exited with code 0. Now the same experiment with a 2-second timeout (SHUTDOWN_TIMEOUT_MS=2000):

export: 200 66194028 bytes 3.345505s curl-exit=18

{"level":"warn","time":"2026-11-14T19:54:10.279Z","service":"bookstore-api","timeoutMs":2000,"msg":"requests still running at the deadline, closing their connections"}
{"level":"warn","time":"2026-11-14T19:54:10.288Z","service":"bookstore-api","requestId":"req-14-export","error":"ERR_STREAM_PREMATURE_CLOSE","durationMs":3350,"msg":"order export aborted"}
{"level":"info","time":"2026-11-14T19:54:10.371Z","service":"bookstore-api","durationMs":2219,"msg":"shutdown complete"}

This time the export was cut, but in a controlled way: the log says what was cut and why, the request.cancel() we wrote in post 13 stopped the SQL query, and the pool closed cleanly. Keep the timeout shorter than the platform's: Kubernetes's default terminationGracePeriodSeconds is 30 seconds and docker stop's default is 10 seconds; when that runs out the process is killed with SIGKILL and no code runs at all.

Why SHUTDOWN_DRAIN_MS? When Kubernetes sends SIGTERM to a pod it also removes the pod from the service endpoints, but the two happen in parallel and the endpoint update can lag. If we call server.close() right away, connections arriving in between are refused. During the drain the server keeps listening and readiness returns 503:

PORT=3001 SHUTDOWN_DRAIN_MS=2000 node src/server.ts

{"status":"ready","checks":{"sql":"ok"}} 200
{"status":"not-ready","checks":{"shutdown":"in progress"}} 503
/health during drain: 200
exit code 0

0 makes sense in development, a few seconds in Kubernetes; choose the value together with the probe intervals from the Kubernetes series. Most readers on Windows will stop the API with Ctrl+C: that sends SIGINT and starts the same shutdown. Windows has no real SIGTERM; SIGTERM is what containers and Linux orchestrators send.

PID 1 in a Container and npm start

The first process in a container becomes PID 1, and the Linux kernel treats PID 1 specially: if there is no handler for a signal, it does not apply the default action (terminating the process). I tried it with a Node script that has no signal handler:

docker run -d --name nodeseries-pid1 node:26-alpine node -e "setInterval(() => {}, 1000)"
docker stop -t 10 nodeseries-pid1

PID 1 without a handler ignored SIGTERM; Docker waited 10 seconds and sent SIGKILL (137 = 128 + 9). --init puts a tiny init process in front, so Node is no longer PID 1 (143 = 128 + 15). Our API has a handler, so being PID 1 does not break its shutdown; still, an init that reaps zombie processes (--init or tini) is a good habit. I tried npm start as well: npm 11 forwarded the SIGTERM it received to its node child and the shutdown lines appeared in the log. Even so, CMD ["node", "src/server.ts"] is preferred in a container: there is no extra process in between and the exit code goes straight to the platform. The Dockerfile itself and image details are in the Docker series.

Scaling: cluster or Replicas?

A Node process runs JavaScript on a single thread (post 01). Node's built-in way to use every CPU core on a machine is node:cluster: a primary process starts several worker processes that listen on the same port.

import cluster from "node:cluster";
import http from "node:http";

const workers = 4;

if (cluster.isPrimary) {
    // The primary only manages workers; it accepts connections and hands them out (round-robin outside Windows)
    console.log(`primary ${process.pid}, scheduling ${cluster.schedulingPolicy === cluster.SCHED_RR ? "round-robin" : "by the OS"}`);
    for (let i = 0; i < workers; i++) {
        cluster.fork();
    }
    let listening = 0;
    cluster.on("listening", async () => {
        if (++listening < workers) {
            return;
        }
        const answeredBy = new Map<string, number>();
        for (let i = 0; i < 8; i++) {
            // agent: false -> a new connection per request, so every request is distributed again
            const pid = await new Promise<string>((resolve) => http.get({ host: "127.0.0.1", port: 3002, agent: false }, (res) => res.setEncoding("utf8").on("data", resolve)));
            answeredBy.set(pid, (answeredBy.get(pid) ?? 0) + 1);
        }
        console.log("requests per worker:", Object.fromEntries(answeredBy));
        cluster.disconnect();
    });
} else {
    // Every worker is a separate process with its own event loop, heap and SQL pool
    http.createServer((req, res) => res.end(String(process.pid))).listen(3002, "127.0.0.1");
}

primary 93238, scheduling round-robin
requests per worker: { '93239': 2, '93240': 2, '93241': 2, '93242': 2 }

Eight requests were spread two per worker across four workers. According to the Node documentation round-robin is the default on every operating system except Windows; on Windows the operating system does the distribution. Workers share no memory: state kept in memory, like the in-memory repositories of post 12, would be separate in each worker. Moving to SQL Server removed that problem, but each worker opens its own SQL pool, so 4 workers × 10 connections.

In the container and Kubernetes world the usual choice is different: one Node process per container, and more replicas for scale. Each replica is monitored, restarted and given CPU and memory limits on its own; graceful shutdown, health checks and logging work exactly as written for a single process. Consider cluster, or PM2 which wraps it, on a single server without an orchestrator (under systemd on a VM, for example). PM2 is a process manager offering restarts, log files and a cluster mode, but in a container the platform already does those jobs.

Measure before scaling: the /diagnostics/event-loop endpoint from post 06 reports the p50/p99 event loop delay. If p99 climbs, find the blocking work first (post 06), then add replicas. Exporting this metric with Prometheus or OpenTelemetry is in the Observability series.

In Production

Validate configuration at startup and refuse to start on errors. Noticing a bad value where it is first used, maybe hours later, is the most expensive kind of failure.

Keep secrets out of log lines. Redaction paths must keep up with the shape of the configuration and headers; when you add a new secret field, update the redaction as well.

Keep liveness simple. Dependency checks belong to readiness; liveness only says whether the process answers.

Nest the shutdown timings: drain + SHUTDOWN_TIMEOUT_MS must be shorter than the platform's limit (terminationGracePeriodSeconds in Kubernetes). Otherwise SIGKILL does not even let the last log line out.

Count connections. Replicas × workers × pool size must stay below what SQL Server can handle.

Common Pitfalls

Production logging with console.log. No levels, no fields, no redaction; and when stdout is a pipe and there is a lot to log, the cost of writing can get in the way of requests.

Not catching SIGTERM. As in Guess First: half a file, unwritten logs, connections left open.

Closing the pool first on shutdown. Requests still running cannot reach the database and return 500; the order is always the HTTP server first, then dependencies.

Calling process.exit() straight away in the handler. It cuts open responses and pino's buffer; wait for the event loop to drain by itself and force the exit only as a last resort.

HOST=127.0.0.1 in a container. The API runs inside the container but port forwarding cannot reach it; use 0.0.0.0 in a container.

Tying readiness to liveness. A database outage sends every pod into a restart loop.

A Small Experiment

★☆☆ Add method and path next to requestId in the logs through mixin. On which lines does this now repeat information, and how would you solve that?

★★☆ Add a rule to loadConfig that rejects a connection string containing TrustServerCertificate=true when NODE_ENV=production (superRefine). Make sure the error message does not show the connection string itself.

★★★ Add "the longest-running request" to the shutdown: keep open requests in a Set in the requestContext middleware and, when the timeout runs out, log in one warn line which paths have been open and for how long.

The API can now be observed, shuts down cleanly and refuses to start with bad configuration. But anyone can still place orders, anyone can read any order, and a script could send thousands of requests per second. In the next post we look at security in Express: helmet, CORS, rate limiting, JWT validation and input limits.