Validating Configuration with JSON Schema: Editor IntelliSense, SchemaStore and Validation in CI

Version note: The examples in this post were run with .NET 10 (JsonSchema.Net 9.7.0), check-jsonschema 0.39.0, Node.js 26.13 (ajv 8.20.0, ajv-formats 3.0.1), Docker Compose 5.1 and actionlint 1.7.12. The specification used is JSON Schema 2020-12.

Guess First

On the staging server of Bookstore, the online bookstore used as the shared example across this site's series, someone edited appsettings.Staging.json by hand to raise the catalog page size to 50:

{
  // edited by hand on the staging server
  "Catalog": {
    "PageSzie": 50,
    "MaxPageSize": "100"
  },
  "Ordering": {
    "InstitutionalMinQuantity": 20,
    "InstitutionalDiscountRate": 0.10
  },
  "Shipping": {
    "DefaultCountry": "no",
    "Carriers": ["standard", "overnight"],
    "SupportEmail": "support-at-bookstore.example"
  }
}

Does Bookstore.Api start with this file? If it does, what is the catalog page size: 50, or the 20 from the base file? Is the string "100" bound to an int property? Who notices the other three mistakes in the file, and when?

Binding Is Not Validation

The code that reads the settings:

#:package Microsoft.Extensions.Configuration.Json@10.0.15
#:package Microsoft.Extensions.Configuration.Binder@10.0.15
#:property PublishAot=false
using Microsoft.Extensions.Configuration;

IConfigurationRoot config = new ConfigurationBuilder()
    .AddJsonFile(Path.GetFullPath("../../bookstore/src/Bookstore.Api/appsettings.json"))
    .AddJsonFile(Path.GetFullPath("appsettings.Staging.json"))
    .Build();

// default binding: unknown keys are ignored, "100" is converted to int
CatalogOptions catalog = config.GetSection("Catalog").Get<CatalogOptions>()!;
Console.WriteLine($"PageSize={catalog.PageSize}, MaxPageSize={catalog.MaxPageSize}");

// the binder can reject keys that have no matching property
try
{
    config.GetSection("Catalog").Get<CatalogOptions>(o => o.ErrorOnUnknownConfiguration = true);
}
catch (InvalidOperationException ex)
{
    Console.WriteLine(ex.Message);
}

class CatalogOptions
{
    public int PageSize { get; set; } = 20;
    public int MaxPageSize { get; set; } = 100;
}

$ dotnet run bind.cs
PageSize=20, MaxPageSize=100
'ErrorOnUnknownConfiguration' was set on the provided BinderOptions, but the following properties were not found on the instance of CatalogOptions: 'PageSzie'

The application started without an error and the page size stayed 20. The binder found no property called PageSzie and ignored that key; since every value in configuration is a string, it also turned "100" into an int without complaint. The person who edited staging believes the change works. ErrorOnUnknownConfiguration catches the unknown key, but only when the application starts and only in the section being bound; it does not catch a lowercase country code like no, a carrier type that is not in the list or a value that does not look like an e-mail address. You can add these checks to the application with ValidateOnStart and DataAnnotations (in the configuration post of the ASP.NET Core Web API series). But the error still shows up at deploy time. To see it earlier, while writing the file, you need a schema.

The Basics of JSON Schema 2020-12

JSON Schema is a language that describes the structure of a JSON document, in JSON itself. The current version is 2020-12; a schema states which version it was written for with $schema. The schema we wrote for Bookstore.Api's own settings sections:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://bookstore.example/schema/bookstore-settings.schema.json",
  "title": "Bookstore.Api settings",
  "description": "The Bookstore-specific sections of appsettings*.json",
  "type": "object",
  "properties": {
    "Catalog": { "$ref": "#/$defs/catalog" },
    "Ordering": { "$ref": "#/$defs/ordering" },
    "Shipping": { "$ref": "#/$defs/shipping" }
  },
  "$defs": {
    "catalog": {
      "type": "object",
      "properties": {
        "PageSize": { "type": "integer", "minimum": 1, "maximum": 100, "default": 20 },
        "MaxPageSize": { "type": "integer", "minimum": 1, "maximum": 500 }
      },
      "additionalProperties": false
    },
    "ordering": {
      "type": "object",
      "properties": {
        "InstitutionalMinQuantity": { "type": "integer", "minimum": 1 },
        "InstitutionalDiscountRate": { "type": "number", "exclusiveMinimum": 0, "maximum": 0.5 }
      },
      "additionalProperties": false
    },
    "shipping": {
      "type": "object",
      "properties": {
        "DefaultCountry": { "type": "string", "pattern": "^[A-Z]{2}$", "description": "ISO 3166-1 alpha-2 country code" },
        "Cutoff": { "type": "string", "pattern": "^([01][0-9]|2[0-3]):[0-5][0-9]$" },
        "Express": { "type": "boolean" },
        "MaxWeightGrams": { "type": "integer", "minimum": 1 },
        "ApiVersion": { "type": "string", "pattern": "^[0-9]+\\.[0-9]+$" },
        "Carriers": {
          "type": "array",
          "items": { "enum": ["standard", "express"] },
          "uniqueItems": true
        },
        "SupportEmail": { "type": "string", "format": "email" }
      },
      "additionalProperties": false
    }
  }
}

The keywords you will use most are in this file. type gives the type of a value, properties the fields of an object, enum the allowed values, pattern a regular expression a string must match, minimum/maximum a numeric range. $defs defines reusable parts inside the schema and $ref points to them. additionalProperties: false treats every key not in the schema as an error; this is the keyword that catches typos. required is deliberately absent: a layer file such as appsettings.Staging.json contains only the settings that change, and a rule like "every file must contain PageSize" breaks layering. Required fields should be checked on the merged configuration, when the application starts.

format is a special case. In 2020-12, format is only an annotation by default, that is, informational; a validator is not required to check the e-mail format. As we will see shortly, tools behave differently here. Let's check that the schema itself is valid and validate the staging file with check-jsonschema. Because appsettings*.json files are JSONC (they contain comments and trailing commas), we pass --force-filetype json5; the JSON5 parser reads JSONC as well:

$ check-jsonschema --check-metaschema ../../bookstore/schema/bookstore-settings.schema.json
ok -- validation done
$ check-jsonschema --force-filetype json5 --schemafile ../../bookstore/schema/bookstore-settings.schema.json appsettings.Staging.json
Schema validation errors were encountered.
  appsettings.Staging.json::$.Catalog.MaxPageSize: '100' is not of type 'integer'
  appsettings.Staging.json::$.Catalog: Additional properties are not allowed ('PageSzie' was unexpected)
  appsettings.Staging.json::$.Shipping.DefaultCountry: 'no' does not match '^[A-Z]{2}$'
  appsettings.Staging.json::$.Shipping.Carriers[1]: 'overnight' is not one of ['standard', 'express']
  appsettings.Staging.json::$.Shipping.SupportEmail: 'support-at-bookstore.example' is not a 'email'

All five mistakes were caught, each with its JSON path. The typo the binder silently skipped is a clear message here.

SchemaStore and Completion in the Editor

The real payoff of JSON Schema is in the editor. SchemaStore is an open catalog of schemas written for common config files; Visual Studio, Visual Studio Code, Rider and other JetBrains IDEs use this catalog. For appsettings.json, package.json, tsconfig.json, compose.yaml and GitHub Actions workflows you don't need to do anything: as soon as the file name matches a pattern in the catalog, completion and error highlighting appear. You can use the same schema on the command line:

$ cat appsettings.loglevel.json
{
  "Logging": {
    "LogLevel": {
      "Default": "Informaton"
    }
  },
  "AllowedHosts": "*"
}
$ check-jsonschema --schemafile https://json.schemastore.org/appsettings.json appsettings.loglevel.json
Schema validation errors were encountered.
  appsettings.loglevel.json::$.Logging.LogLevel.Default: 'Informaton' is not one of ['Trace', 'Debug', 'Information', 'Warning', 'Error', 'Critical', 'None']

The list of LogLevel values comes from the appsettings schema in SchemaStore; you see the same list as completions in the editor. There are two ways to introduce your own schema to the editor. The first is writing "$schema": "..." inside the file; the Visual Studio Code documentation reminds you that this changes the JSON itself and can cause trouble in other systems that read the file (in .NET this key also ends up in configuration). The second is mapping it by file pattern in the workspace settings; that is what we chose for Bookstore:

{
  "json.schemas": [
    {
      "fileMatch": ["appsettings*.json"],
      "url": "./schema/bookstore-settings.schema.json"
    }
  ]
}

With this mapping the editor checks appsettings*.json files against both SchemaStore's appsettings schema and ours. Visual Studio and Rider have their own schema mapping settings; I did not run these two IDEs for this post, the steps are in the products' documentation.

On the YAML side, Red Hat's YAML extension (yaml-language-server) uses SchemaStore and lets you choose a schema per file with a comment. The YAML version of Bookstore's shipping settings, with the schema comment:

# yaml-language-server: $schema=../../bookstore/schema/bookstore-settings.schema.json
Shipping:
  DefaultCountry: "NO"
  Cutoff: "12:30"
  Express: true
  MaxWeightGrams: 1000
  ApiVersion: "1.10"
  Carriers:
    - standard
    - express

$ check-jsonschema --schemafile ../../bookstore/schema/bookstore-settings.schema.json shipping.yaml ../04-config/settings/shipping.yaml
Schema validation errors were encountered.
  ../04-config/settings/shipping.yaml::$.Shipping.Express: 'yes' is not of type 'boolean'
  ../04-config/settings/shipping.yaml::$.Shipping.ApiVersion: 1.1 is not of type 'string'

The correctly written file passed (check-jsonschema prints no separate line for files that pass). The unquoted YAML from the fourth post of this series produced two errors: yes is a string, not a boolean, and 1.10 is a number, not a string. The 1.10 that yamllint could not catch in the third post was caught by the schema, because the schema knows the value's type. check-jsonschema reads YAML with the 1.2 rules; that is why NO caused no trouble.

Ready-Made Schemas for Compose and Workflow Files

check-jsonschema carries some of SchemaStore's schemas inside it; you can use them without going online. Let's misspell environment in a Compose file:

$ cat compose.typo.yaml
services:
  api:
    image: bookstore-api:dev
    enviroment:
      Catalog__PageSize: "20"
$ check-jsonschema --builtin-schema vendor.compose-spec compose.typo.yaml
Schema validation errors were encountered.
  compose.typo.yaml::$.services.api: Unevaluated properties are not allowed ('enviroment' was unexpected)
$ docker compose -f compose.typo.yaml config 2>&1 | sed "s|$PWD/||"
validating compose.typo.yaml: services.api additional properties 'enviroment' not allowed

The schema and docker compose config found the same error; Compose does its own validation with the same Compose Specification schema. The difference is that the schema runs in a CI step without Docker installed, in under a second.

Mixing Drafts

The versions of JSON Schema (draft-04, draft-06, draft-07, 2019-09, 2020-12) differ in their keywords. A validator looks at $schema, applies that version's keywords and skips keywords it does not know without an error. Imagine adding unevaluatedProperties, introduced in 2019-09, to a draft-07 schema copied from the internet:

$ cat catalog.draft-07.schema.json
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "Catalog": {
      "type": "object",
      "properties": {
        "PageSize": { "type": "integer" }
      },
      "unevaluatedProperties": false
    }
  }
}
$ check-jsonschema --force-filetype json5 --schemafile catalog.draft-07.schema.json appsettings.Staging.json
ok -- validation done
$ check-jsonschema --force-filetype json5 --schemafile catalog.2020-12.schema.json appsettings.Staging.json
Schema validation errors were encountered.
  appsettings.Staging.json::$.Catalog: Unevaluated properties are not allowed ('MaxPageSize', 'PageSzie' were unexpected)

The same schema, with only the $schema line different, caught nothing under draft-07 and caught two extra properties under 2020-12. The nasty part of this pitfall is that it reports "ok", not an error. When writing a schema, always set $schema explicitly to 2020-12, check which version the parts you copy belong to, and check the schema with --check-metaschema. OpenAPI 3.1 is aligned with JSON Schema 2020-12 too; that is why the schemas in the OpenAPI document .NET 10 produces use the same keywords (in the .NET 10 changes post of the ASP.NET Core Web API series).

One Schema, Three Validators

You can also run the schema in Node.js with ajv and in .NET with JsonSchema.Net. ajv has a separate CLI package (ajv-cli), but its last release is from 2021; a small script that uses the library directly is a more current choice (run npm install in the folder first):

$ node validate.mjs ../../bookstore/schema/bookstore-settings.schema.json appsettings.Staging.json
appsettings.Staging.json: /Catalog must NOT have additional properties
appsettings.Staging.json: /Catalog/MaxPageSize must be integer
appsettings.Staging.json: /Shipping/DefaultCountry must match pattern "^[A-Z]{2}$"
appsettings.Staging.json: /Shipping/Carriers/1 must be equal to one of the allowed values
appsettings.Staging.json: /Shipping/SupportEmail must match format "email"

#:package JsonSchema.Net@9.7.0
#:property PublishAot=false
using System.Text.Json;
using Json.Schema;

JsonSchema schema = JsonSchema.FromFile(Path.GetFullPath(args[0]));

// appsettings files are JSONC: skip comments, allow trailing commas
JsonDocumentOptions jsonc = new() { CommentHandling = JsonCommentHandling.Skip, AllowTrailingCommas = true };
using JsonDocument data = JsonDocument.Parse(File.ReadAllText(args[1]), jsonc);

EvaluationResults results = schema.Evaluate(data.RootElement, new EvaluationOptions { OutputFormat = OutputFormat.List });
Console.WriteLine($"{args[1]}: {(results.IsValid ? "valid" : "invalid")}");
foreach (EvaluationResults detail in (results.Details ?? []).Where(d => d.Errors is { Count: > 0 }))
{
    foreach ((string keyword, string message) in detail.Errors!)
    {
        Console.WriteLine($"  {detail.InstanceLocation} [{keyword}] {message}");
    }
}

$ dotnet run validate.cs -- ../../bookstore/schema/bookstore-settings.schema.json appsettings.Staging.json
appsettings.Staging.json: invalid
   [properties] Some properties did not match the required schema: ["Catalog","Shipping"]
  /Catalog [properties] Some properties did not match the required schema: ["MaxPageSize"]
  /Catalog [additionalProperties] Some additional properties did not match the required schema: ["PageSzie"]
  /Shipping [properties] Some properties did not match the required schema: ["DefaultCountry","Carriers"]
  /Catalog/MaxPageSize [type] Value is "string" but should be "integer"
  /Catalog/PageSzie [] All values fail against the false schema
  /Shipping/DefaultCountry [pattern] The string value is not a match for the indicated regular expression
  /Shipping/Carriers [items] Some items do not match the required schema; failing indexes: [1]
  /Shipping/Carriers/1 [enum] Value should match one of the values specified by the enum

The three validators found the same errors, with two differences. The first is the shape of the messages: check-jsonschema and ajv give a flat list, JsonSchema.Net returns a result for every schema level. The second is format: ajv checked the e-mail once ajv-formats was added; JsonSchema.Net did not with default settings, because format is an annotation in 2020-12. If a check really has to happen, don't rely on format; write a pattern or switch on the validator's format checking explicitly. On the .NET side, putting this code in a unit test is the easiest way to validate the schema and the settings files in the same test run.

Validation in CI

We added two steps to the lint job of Bookstore's CI workflow:

name: ci

on:
  push:
    branches: [main]
  pull_request:

permissions:
  contents: read

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-python@v7
        with:
          python-version: "3.14"
      - run: pip install yamllint==1.38.0 check-jsonschema==0.39.0 json5==0.17.3
      # -f github turns every finding into an annotation on the pull request
      - run: yamllint -f github .
      - name: Validate settings against Bookstore's schema
        run: >-
          check-jsonschema --force-filetype json5
          --schemafile schema/bookstore-settings.schema.json
          src/Bookstore.Api/appsettings*.json
      - name: Validate Compose and workflow files
        run: |
          check-jsonschema --builtin-schema vendor.compose-spec compose.yaml
          check-jsonschema --builtin-schema vendor.github-workflows .github/workflows/*.yml

  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-dotnet@v6
        with:
          dotnet-version: 10.0.x
      - name: Build
        run: |
          dotnet restore src/Bookstore.Api
          dotnet build src/Bookstore.Api -c Release --no-restore

$ check-jsonschema --force-filetype json5 --schemafile schema/bookstore-settings.schema.json src/Bookstore.Api/appsettings*.json
ok -- validation done
$ check-jsonschema --builtin-schema vendor.compose-spec compose.yaml
ok -- validation done
$ check-jsonschema --builtin-schema vendor.github-workflows .github/workflows/*.yml
ok -- validation done
$ actionlint .github/workflows/ci.yml && echo "actionlint: no findings"
actionlint: no findings

We used >- to split the long command over lines: because a folded scalar joins lines with spaces, it becomes a single command, the correct use of the pitfall from the third post of this series. We pinned the package versions; the SchemaStore schemas bundled in check-jsonschema change with its version, and a fixed version keeps the CI result from turning red one day for no reason.

You can move the same check even earlier, to commit time: the check-jsonschema documentation has ready-made hook definitions for the pre-commit framework. But hooks can be skipped on a developer's machine; the CI step is the only place where the rule applies to everyone. In Bookstore we use both: the hook gives fast feedback, CI makes the final call. If an operations team edits staging or production files by hand, the same command helps them too: the error message says what is wrong in which file and at which JSON path, without having to read the application log.

In Production

A schema is a contract; if it is not updated when the application code changes, it produces false alarms or misses errors. In Bookstore the rule is that a pull request adding a new options property also updates the schema. The way not to forget is to compare the schema with the options classes in a unit test, or to generate the schema from code; JsonSchemaExporter, introduced in .NET 9, can produce a JSON Schema from a type. Expect to add rules such as pattern and enum to the generated schema by hand.

Don't apply additionalProperties: false to a framework's own sections (Logging, Kestrel); their schema lives in SchemaStore and changes with versions. Keep your own sections closed. If you use a remote schema URL (like SchemaStore), be aware that CI needs network access and that the version of that schema can change; copying critical schemas into the repo pins the result.

Common Pitfalls

The most common is never writing additionalProperties: by default every extra key is allowed, so the schema never catches typos. The second is using additionalProperties: false in schemas combined with allOf: this keyword only sees the properties of the schema it is in and counts fields coming from the other part as "additional". unevaluatedProperties in 2020-12 exists for this case.

The third is draft confusion: using $defs, prefixItems or unevaluatedProperties in a draft-07 schema and having them silently ignored. The fourth is assuming format is checked by every validator. Finally, trying to validate JSONC files with a plain JSON parser: at the first comment in appsettings.json the validator cannot parse the file and fails without ever looking at the schema.

A Small Experiment

★☆☆ Add the Shipping section to Bookstore.Api's appsettings.json and deliberately write "Carriers": ["standard", "standard"]. Which keyword catches it?

★★☆ Add a rule for Ordering to the schema: if InstitutionalDiscountRate is given, InstitutionalMinQuantity must be given too. Use the dependentRequired keyword from 2020-12 and try the same rule with a draft-07 $schema.

★★★ Generate a schema from the CatalogOptions class with JsonSchemaExporter and compare it with the catalog part of the schema we wrote by hand. Which rules could be derived from the code, and which could not?

We have seen validation with schemas while writing files and in CI. In the last post of the series we move to the command line: querying, transforming and editing JSON in place with jq and YAML with yq, from updating the image tag in Bookstore's manifest to writing a condition with jq in a GitHub Actions step.