WebApplicationFactory in Depth: Custom Factory, ConfigureTestServices, Configuration and Test Authentication

Version note: The code in this post was built and run with .NET 10, xUnit.net v3 4.0.3, Microsoft.AspNetCore.Mvc.Testing 10.0.15 and Testcontainers for .NET 4.16.

Guess First

You want to test Bookstore's payment integration. The plan is simple: the fake payment server we will set up in post 4 listens on http://localhost:9091/, and in the test you point the Payments:BaseUrl setting at that address. Someone on the team registered the typed client in Program.cs like this:

Uri paymentsBaseUrl = new(builder.Configuration["Payments:BaseUrl"]!);
builder.Services.AddHttpClient<PaymentClient>(client => client.BaseAddress = paymentsBaseUrl)
    .AddStandardResilienceHandler(builder.Configuration.GetSection("Payments:Resilience"));

I tried four ways of giving the setting to the test factory. Each factory is a separate class, and they all provide the same value:

public sealed class UseSettingFactory : ProbeFactory
{
    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        base.ConfigureWebHost(builder);
        builder.UseSetting("Payments:BaseUrl", FakeUrl);
    }
}

public sealed class AppConfigurationFactory : ProbeFactory
{
    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        base.ConfigureWebHost(builder);
        builder.ConfigureAppConfiguration(config =>
            config.AddInMemoryCollection(new Dictionary<string, string?> { ["Payments:BaseUrl"] = FakeUrl }));
    }
}

public sealed class HostConfigurationFactory : ProbeFactory
{
    protected override IHost CreateHost(IHostBuilder builder)
    {
        builder.ConfigureHostConfiguration(config =>
            config.AddInMemoryCollection(new Dictionary<string, string?> { ["Payments:BaseUrl"] = FakeUrl }));
        return base.CreateHost(builder);
    }
}

// The fourth: an environment variable set before the factory is created
Environment.SetEnvironmentVariable("Payments__BaseUrl", ProbeFactory.FakeUrl);

After the host starts, ProbeFactory looks at two things: the BaseAddress of the HttpClient that IHttpClientFactory creates for PaymentClient, and the value inside IOptions<PaymentOptions>. Do all four methods show the fake address in options? And in which of them does paymentsBaseUrl, read early in Program.cs, become the fake address?

The Answer: When Does Configuration Reach Program.cs?

The real output of the experiment:

UseSettingFactory: eager=http://localhost:9091/ options=http://localhost:9091/
AppConfigurationFactory: eager=https://api.payments.example/ options=http://localhost:9091/
HostConfigurationFactory: eager=http://localhost:9091/ options=http://localhost:9091/
EnvironmentVariable: eager=http://localhost:9091/ options=http://localhost:9091/

Options saw the right address in all four cases. But the value given with ConfigureAppConfiguration did not reach the place Program.cs reads before builder.Build(): the typed client was set up to call the real payment address from appsettings.json. Even if the test stays green, that means the test tries to send requests to the production payment provider.

The integration testing page on Microsoft Learn draws this line explicitly. A source added with ConfigureAppConfiguration is preserved but only available after the app is built. Values given with ConfigureHostConfiguration inside CreateHost are read before the entry point runs and passed in as args; that is why Program.cs must pass args to WebApplication.CreateBuilder(args). Without args these settings have no effect at all. Bookstore's Program.cs passes args.

The environment variable works too, but it is a process-wide setting. xUnit v3 runs different test classes in parallel; if one class changes the environment variable while another class's factory is starting, which value it sees is undefined. UseSetting, which is scoped to the factory, is safer for that reason.

The real lesson is on the Program.cs side. Reading configuration into a local variable before Build() freezes it, no matter who changes it later or when. Bookstore's real Program.cs reads the address from options, through the service provider, at the moment the client is created:

builder.Services.AddOptions<PaymentOptions>().BindConfiguration("Payments");

builder.Services.AddHttpClient<PaymentClient>((sp, client) =>
    {
        // The address is read when the client is created; it sees the test's configuration
        PaymentOptions payments = sp.GetRequiredService<IOptions<PaymentOptions>>().Value;
        client.BaseAddress = payments.BaseUrl;
        client.DefaultRequestHeaders.Add("X-Api-Key", payments.ApiKey);
    })
    .AddStandardResilienceHandler(builder.Configuration.GetSection("Payments:Resilience"));

In this form all four methods work. .NET 11 adds a new builder hook to the factory for this table; the details are in the last post of the series, together with the differences between versions.

The Final Custom Factory

The factory in post 1 only changed a connection string. By the end of this post it looks like this:

using Bookstore.Api.IntegrationTests.Infrastructure;
using Microsoft.AspNetCore.Authentication;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Mvc.Testing;
using Microsoft.AspNetCore.TestHost;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;

namespace Bookstore.Api.IntegrationTests;

public sealed class BookstoreApiFactory(SqlServerFixture sql) : WebApplicationFactory<Program>
{
    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        builder.UseEnvironment("Testing");
        builder.UseSetting("ConnectionStrings:Bookstore", sql.ConnectionString);

        builder.ConfigureTestServices(services =>
        {
            // A test user instead of JWT; this line also changes the default scheme
            services.AddAuthentication(TestAuthHandler.SchemeName)
                .AddScheme<AuthenticationSchemeOptions, TestAuthHandler>(TestAuthHandler.SchemeName, _ => { });

            // Remove the background service we don't want running in tests
            ServiceDescriptor worker = services.Single(d =>
                d.ServiceType == typeof(IHostedService) && d.ImplementationType == typeof(LowStockReportService));
            services.Remove(worker);
        });
    }

    public HttpClient CreateClientFor(TestUser user, bool allowAutoRedirect = true)
    {
        HttpClient client = CreateClient(new WebApplicationFactoryClientOptions { AllowAutoRedirect = allowAutoRedirect });
        client.DefaultRequestHeaders.Add(TestAuthHandler.UserHeader, user.HeaderValue);
        return client;
    }
}

There are three parts: configuration (UseEnvironment, UseSetting), service changes (ConfigureTestServices) and a helper that makes tests easier to read (CreateClientFor). Setting the environment name to Testing keeps developer settings in appsettings.Development.json out of the tests; WebApplicationFactory assumes Development unless you change it.

Replacing Services: ConfigureServices or ConfigureTestServices?

Older blog posts often state this rule: the factory's ConfigureServices runs before the app's own registrations, so replace the service you want in ConfigureTestServices. The rule comes from the older hosting model that uses a Startup class; the Learn page also notes that in that model the test ConfigureServices runs before Startup.ConfigureServices. I measured the situation in a .NET 10 app with minimal hosting. I replaced Program's AddSingleton(TimeProvider.System) registration and the JWT default scheme in two separate factories, one with ConfigureServices and the other with ConfigureTestServices:

ConfigureServices: TimeProvider=FakeTimeProvider, POST /api/books as admin -> 201
ConfigureTestServices: TimeProvider=FakeTimeProvider, POST /api/books as admin -> 201

Both ran after Program.cs; the last registration won and the test user could add a book as admin. I ran the same experiment for a factory that registers one clock in ConfigureTestServices while the test registers another through WithWebHostBuilder; the test's clock won with both methods. So in .NET 10 mixing up the order is not as dangerous as it used to be. I still prefer ConfigureTestServices: the intent is clear ("I'm changing this only for tests") and it keeps the same meaning if the code moves to a project with a Startup class.

What you really need to watch is that "the last registration wins" does not hold everywhere. A GetRequiredService<TimeProvider>() that asks for a single instance gets the last registration. But for services resolved as IEnumerable<T>, a new registration does not replace the old one, it is added next to it. Hosted services are resolved exactly this way; that is why the factory does not replace LowStockReportService with something new but finds its registration and removes it.

Test Authentication: Telling 401 and 403 Apart

In production Bookstore expects a JWT bearer token. Getting a token from a real identity provider in tests is slow and fragile, and validating the token is the framework's job anyway. For tests we write an authentication handler that builds the user from a header on the request:

using System.Security.Claims;
using System.Text.Encodings.Web;
using Microsoft.AspNetCore.Authentication;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using Microsoft.Extensions.Primitives;

namespace Bookstore.Api.IntegrationTests.Infrastructure;

public sealed class TestAuthHandler(
    IOptionsMonitor<AuthenticationSchemeOptions> options, ILoggerFactory logger, UrlEncoder encoder)
    : AuthenticationHandler<AuthenticationSchemeOptions>(options, logger, encoder)
{
    public const string SchemeName = "Test";
    public const string UserHeader = "X-Test-User";

    protected override Task<AuthenticateResult> HandleAuthenticateAsync()
    {
        // No header means an anonymous request: NoResult, which produces 401 on challenge
        if (!Request.Headers.TryGetValue(UserHeader, out StringValues header))
        {
            return Task.FromResult(AuthenticateResult.NoResult());
        }

        // Header format: "<customerId>;<role>,<role>"
        string[] parts = header.ToString().Split(';');
        List<Claim> claims = [new("customer_id", parts[0]), new(ClaimTypes.Name, $"customer-{parts[0]}")];
        if (parts.Length > 1)
        {
            claims.AddRange(parts[1].Split(',', StringSplitOptions.RemoveEmptyEntries)
                .Select(role => new Claim(ClaimTypes.Role, role)));
        }

        ClaimsIdentity identity = new(claims, SchemeName);
        AuthenticationTicket ticket = new(new ClaimsPrincipal(identity), SchemeName);
        return Task.FromResult(AuthenticateResult.Success(ticket));
    }
}

namespace Bookstore.Api.IntegrationTests.Infrastructure;

public sealed record TestUser(int CustomerId, params string[] Roles)
{
    public static TestUser Customer(int id) => new(id);

    public static TestUser Admin { get; } = new(1, "admin");

    public string HeaderValue => $"{CustomerId};{string.Join(',', Roles)}";
}

The sample handler on Learn authenticates every request with the same user. That is enough for the "signed-in user" scenario but makes it impossible to test an anonymous request. Returning NoResult when there is no header lets you test all three cases with the same factory: no header, a header without the role, and a header with the right role. Because services.AddAuthentication(TestAuthHandler.SchemeName) changes the default scheme, challenge and forbid also land on this handler; the JWT handler stays registered but never kicks in.

[Fact]
public async Task Anonymous_user_gets_401()
{
    HttpClient client = factory.CreateClient();

    HttpResponseMessage response = await client.PostAsJsonAsync("/api/books", NewBook("978-0321125217"),
        TestContext.Current.CancellationToken);

    ProblemDetails problem = await response.AssertProblemAsync(HttpStatusCode.Unauthorized);
    Assert.Equal("Unauthorized", problem.Title);
}

[Fact]
public async Task Customer_without_admin_role_gets_403()
{
    HttpClient client = factory.CreateClientFor(TestUser.Customer(42));

    HttpResponseMessage response = await client.PostAsJsonAsync("/api/books", NewBook("978-0321125217"),
        TestContext.Current.CancellationToken);

    ProblemDetails problem = await response.AssertProblemAsync(HttpStatusCode.Forbidden);
    Assert.Equal("Forbidden", problem.Title);
}

[Fact]
public async Task Admin_creates_book_and_gets_its_location()
{
    HttpClient client = factory.CreateClientFor(TestUser.Admin);

    HttpResponseMessage response = await client.PostAsJsonAsync("/api/books", NewBook("978-0321125218"),
        TestContext.Current.CancellationToken);

    int id = response.AssertCreatedAt("/api/books/");
    BookDto? created = await client.GetFromJsonAsync<BookDto>($"/api/books/{id}", TestContext.Current.CancellationToken);
    Assert.Equal("978-0321125218", created!.Isbn);
}

401 means "I don't know who you are", 403 means "I know who you are, but you aren't allowed to do this". Telling them apart matters because authorization bugs usually appear right at that line: a policy gets attached to the wrong endpoint, or only RequireAuthorization() is used instead of a role check and every signed-in customer becomes a catalog admin. Testing every combination of roles and claims as a matrix is the subject of posts 1 and 10 of the API Security series; writing these three tests for each policy is a reasonable baseline here.

Small Helpers for Redirects, ProblemDetails and Location

Bookstore's old catalog links use the ISBN and are redirected to the canonical address with a 301. The default test client follows redirects, so this behavior is invisible; in a run without AllowAutoRedirect = false, the request ended with 200 at http://localhost/api/books/1. To verify the redirect itself:

[Fact]
public async Task Old_isbn_link_redirects_permanently_to_canonical_url()
{
    HttpClient admin = factory.CreateClientFor(TestUser.Admin);
    HttpResponseMessage created = await admin.PostAsJsonAsync("/api/books", NewBook("978-0321125220"),
        TestContext.Current.CancellationToken);
    int id = created.AssertCreatedAt("/api/books/");

    HttpClient client = factory.CreateClientFor(TestUser.Customer(42), allowAutoRedirect: false);
    HttpResponseMessage response = await client.GetAsync("/books/isbn/978-0321125220", TestContext.Current.CancellationToken);

    Assert.Equal(HttpStatusCode.MovedPermanently, response.StatusCode);
    Assert.Equal($"/api/books/{id}", response.Headers.Location?.OriginalString);
}

The helpers used by the tests above collect three checks that repeat in every test: the status code, the application/problem+json content type and reading the body as the right type.

public static class HttpAssertions
{
    public static async Task<HttpValidationProblemDetails> AssertValidationProblemAsync(this HttpResponseMessage response)
    {
        Assert.Equal(HttpStatusCode.BadRequest, response.StatusCode);
        Assert.Equal("application/problem+json", response.Content.Headers.ContentType?.MediaType);
        HttpValidationProblemDetails? problem = await response.Content.ReadFromJsonAsync<HttpValidationProblemDetails>();
        return Assert.IsType<HttpValidationProblemDetails>(problem);
    }

    public static async Task<ProblemDetails> AssertProblemAsync(this HttpResponseMessage response, HttpStatusCode expected)
    {
        Assert.Equal(expected, response.StatusCode);
        Assert.Equal("application/problem+json", response.Content.Headers.ContentType?.MediaType);
        ProblemDetails? problem = await response.Content.ReadFromJsonAsync<ProblemDetails>();
        return Assert.IsType<ProblemDetails>(problem);
    }

    public static int AssertCreatedAt(this HttpResponseMessage response, string pathPrefix)
    {
        // 201 + Location: /api/books/{id}; return the id for the rest of the test
        Assert.Equal(HttpStatusCode.Created, response.StatusCode);
        string location = Assert.IsType<Uri>(response.Headers.Location).OriginalString;
        Assert.StartsWith(pathPrefix, location);
        return int.Parse(location[pathPrefix.Length..]);
    }
}

401 and 403 also come back with a ProblemDetails body thanks to AddProblemDetails and UseStatusCodePages. The helper rejects a bare status code without a body; that is deliberate, because the API's error contract is something the Angular side relies on too. The real bodies:

401 application/problem+json {"type":"https://tools.ietf.org/html/rfc9110#section-15.5.2","title":"Unauthorized","status":401,"traceId":"00-606b8a27aac446e1aaaebc295dcc9a8b-8eb04a050b6d1171-00"}
403 application/problem+json {"type":"https://tools.ietf.org/html/rfc9110#section-15.5.4","title":"Forbidden","status":403,"traceId":"00-73fbd0e74ebe264f31fcca5caeb8ce0a-4183051e61f0d921-00"}
400 application/problem+json {"type":"https://tools.ietf.org/html/rfc9110#section-15.5.1","title":"One or more validation errors occurred.","status":400,"errors":{"price":["Price must be greater than zero."],"title":["Title is required."]},"traceId":"00-1c22ebd95a255b216ed871d63a8e5fdb-ea4d189f30d68eb5-00"}

The validation test looks at the errors keys in that body:

HttpValidationProblemDetails problem = await response.AssertValidationProblemAsync();
Assert.Equal(["price", "title"], problem.Errors.Keys.Order());

Detailed output of the whole suite (paths shortened):

$ dotnet test --project tests/Bookstore.Api.IntegrationTests --output detailed
passed Bookstore.Api.IntegrationTests.CatalogAuthorizationTests.Old_isbn_link_redirects_permanently_to_canonical_url (310ms)
passed Bookstore.Api.IntegrationTests.CatalogAuthorizationTests.Invalid_book_returns_validation_problem (13ms)
passed Bookstore.Api.IntegrationTests.CatalogAuthorizationTests.Customer_without_admin_role_gets_403 (2ms)
passed Bookstore.Api.IntegrationTests.CatalogAuthorizationTests.Admin_creates_book_and_gets_its_location (29ms)
passed Bookstore.Api.IntegrationTests.CatalogAuthorizationTests.Anonymous_user_gets_401 (1ms)
passed Bookstore.Api.IntegrationTests.BookCatalogTests.Unknown_book_returns_404_problem_details (370ms)
passed Bookstore.Api.IntegrationTests.BookEditionTests.Second_edition_is_listed_with_its_own_price (379ms)
passed Bookstore.Api.IntegrationTests.BookCatalogTests.Search_is_case_insensitive_like_production (10ms)
  total: 8
  failed: 0
  succeeded: 8
  duration: 12s 662ms

The first test of each class takes over 300 milliseconds, the rest a few milliseconds. The difference is the host starting up; that is exactly the topic of the next section.

Sharing the Factory and the Cost of WithWebHostBuilder

I measured the cost of creating a factory per test, getting a new client from the same factory, and producing a variation with WithWebHostBuilder. Each round sent the same request:

using WebApplicationFactory<Program> variant = factory.WithWebHostBuilder(b =>
    b.ConfigureTestServices(s => s.AddSingleton<TimeProvider>(new FakeTimeProvider())));
await variant.CreateClient().GetAsync("/api/books/1", TestContext.Current.CancellationToken);

round 1: new factory 262 ms, same factory 17 ms, WithWebHostBuilder 55 ms
round 2: new factory 42 ms, same factory 1 ms, WithWebHostBuilder 46 ms
round 3: new factory 51 ms, same factory 1 ms, WithWebHostBuilder 43 ms

The first round includes JIT warm-up. In the later rounds the picture is clear: getting a client from the same factory takes 1 millisecond, while WithWebHostBuilder is as expensive as a new factory, because behind the scenes it builds and starts a new host. With Bookstore's small Program.cs that is around 45 milliseconds; it grows as DI registrations and startup work grow. If each of 300 tests uses WithWebHostBuilder, that is 10–15 seconds of extra time regardless of the tests themselves. The rule: use WithWebHostBuilder when a variation really needs a different registration, and a new client from the same factory when it only needs a different user or different data.

xUnit v3 offers two options for the scope of sharing. IClassFixture<BookstoreApiFactory> gives each test class its own host; classes can run in parallel and each host is independent. Making the factory an assembly fixture like the container, and running the whole suite on a single host, looks tempting; when I tried it, the build stopped with this error:

error xUnit3005: Type 'Bookstore.Api.IntegrationTests.BookstoreApiFactory' must have a non-obsolete public constructor: public Bookstore.Api.IntegrationTests.BookstoreApiFactory([ITestContextAccessor accessor], [ITestOutputHelper helper], [IMessageSink diagnosticMessageSink])

An assembly fixture's constructor cannot take another fixture; it can only take xUnit's own services. If you want a single host, you have to manage the container and the factory inside the same assembly fixture class. In Bookstore I stay with class fixtures: we pay one host per test class (around 45 milliseconds here), and in exchange a change one class makes to its host does not leak into another.

In Production

The test authentication handler should live only in the test project. The moment the API project gets code like "read the user from a header in the Testing environment", a build that reaches production with the wrong environment variable could give everyone admin rights. Registering the handler inside ConfigureTestServices guarantees this code never enters the production assembly.

JWT validation itself stayed outside these tests: issuer, audience, signature and lifetime checks. Those need a few separate tests with tokens produced by a real signing key; how many, and at which level, is covered in the API Security series.

Catch code that reads configuration early in code review. If the result of a builder.Configuration[...] call is assigned to a local variable and carried into a lambda, the value a test sees depends on which method the factory uses. The options pattern and late reads through the service provider remove that uncertainty.

Common Pitfalls

A new factory per test. Writing using BookstoreApiFactory factory = new(sql); at the top of a test method feels fully isolated, but it adds the 40–50 millisecond host cost from the measurement to every test and still shares the database. The real isolation problem is in the data; that is the topic of post 3.

Background services running in tests without you meaning them to. In the first version of the factory, LowStockReportService was registered and I applied migrations after the host had started. The service tried to connect to the database before the migration:

fail: Microsoft.Extensions.Hosting.Internal.Host[9]
      BackgroundService failed
      System.InvalidOperationException: An exception has been raised that is likely due to a transient failure. Consider enabling transient error resiliency by adding 'EnableRetryOnFailure' to the 'UseSqlServer' call.
       ---> Microsoft.Data.SqlClient.SqlException (0x80131904): Cannot open database "BookstoreDb" requested by the login. The login failed.
      Login failed for user 'sa'.
crit: Microsoft.Extensions.Hosting.Internal.Host[10]
      The HostOptions.BackgroundServiceExceptionBehavior is configured to StopHost. A BackgroundService has thrown an unhandled exception, and the IHost instance is stopping. To avoid this behavior, configure this to Ignore; however the BackgroundService will not be restarted.

The host stopped and both tests failed with an unrelated error: System.ObjectDisposedException : Cannot access a disposed object. Object name: 'Microsoft.AspNetCore.TestHost.TestServer'. Someone reading that error looks for a problem in the tests; the real cause is a background service the tests don't care about at all. Decide for every hosted service that runs on a timer, consumes messages or calls out: if the tests don't need it, remove its registration; if they do, write a separate test class for it and control time with FakeTimeProvider.

Adding the test handler without changing the default scheme. If you only call AddScheme and not AddAuthentication(TestAuthHandler.SchemeName), the default scheme stays JWT bearer; the test handler is registered, but no request reaches it. I tried it: the admin request got 401 with WWW-Authenticate: Bearer. If policies name a scheme explicitly (AuthenticationSchemes = "Bearer"), you need to register the test handler under the same name.

Trusting ConfigureAppConfiguration and believing you changed a value that is read early. The table in Guess First shows this is a silent bug: the test can stay green, because the request trying to reach the real address may get some answer or may never be sent.

A Small Experiment

★☆☆ Change TestAuthHandler to return AuthenticateResult.Fail("no user") when there is no header. Does Anonymous_user_gets_401 still pass? What changes for GET /api/books, which is open to anonymous access?

★★☆ Add a "the user must have a catalog:write claim" requirement to the CatalogAdmin policy. Extend TestUser to carry claims and write a test that produces 403 only when the claim is missing.

★★★ Build a setup that manages the container and the factory in a single assembly fixture. Compare the suite duration with the class fixture version, then find a scenario where two classes running in parallel on the same host can affect each other (hint: a test that advances FakeTimeProvider).

The factory now controls users, configuration and services. But all test classes still share the same database, and the order dependency from post 1 is not solved. In the next post we look at the Testcontainers lifecycle, migrations and three ways of cleaning up between tests: Respawn, transaction rollback and a database per test.