Keeping Test Code Clean: Test Data Builders, Object Mothers, AutoFixture and Bogus

Version note: The code in this post was built and run with .NET 10, xUnit.net v3 4.0.2, AutoFixture 4.18.1, AutoFixture.AutoNSubstitute 4.18.1, AutoFixture.Xunit3 4.19.0, Bogus 35.6.5 and NSubstitute 6.2.0.

Predict First

Bookstore's shipping rule: domestic orders in Turkey below 50 EUR pay 4.99 EUR shipping, 50 and above ship free, international shipping is 12 EUR. Saying "fixed data blinds tests", a developer wrote the shipping test with random data from Bogus:

namespace Bookstore.Ordering;

public static class ShippingPolicy
{
    public static decimal FeeFor(Order order) =>
        order.Customer.ShippingAddress.Country != "TR" ? 12.00m
        : order.Subtotal >= 50m ? 0m
        : 4.99m;
}

[Fact]
public void Domestic_order_below_free_shipping_threshold_pays_4_99()
{
    Faker faker = new();
    Customer customer = new("C-1", faker.Name.FullName(), faker.Internet.Email(),
        new Address(faker.Address.StreetAddress(), "Istanbul", "34000", "TR"), 0);
    Book book = new(faker.Commerce.Ean13(), faker.Commerce.ProductName(), faker.Random.Decimal(5m, 60m));
    Order order = new(Guid.NewGuid(), customer, [new OrderLine(book, 1)]);

    Assert.Equal(4.99m, ShippingPolicy.FeeFor(order));
}

I ran the test 30 times. How many times did it fail? When it fails, does the failure message say what was wrong?

The Answer: 6 Red Runs Out of 30

pass=24 fail=6

    Bookstore.Ordering.Tests.RandomDataTests.Domestic_order_below_free_shipping_threshold_pays_4_99 [FAIL]
      Assert.Equal() Failure: Values differ
      Expected: 4.99
      Actual:   0

The book price is random between 5 and 60; when the price goes above 50, shipping becomes free and the test fails. The code is right, the test is wrong: the test name says "an order below the threshold", but the data does not guarantee it. The failure message does not show the price either; someone who sees the red in CI (Continuous Integration) cannot reproduce it. Random data is a tool; used wrongly, it produces flaky tests. In this post we look at four ways of building test data, along with when each one helps.

Test Data Builder: Meaningful Defaults

The Test Data Builder, written up by Nat Pryce in 2007 and also described in GOOS (Growing Object-Oriented Software, Guided by Tests), is a small class that builds a valid object with meaningful defaults and lets the test change only the field it cares about:

// Test Data Builder: meaningful defaults, the test changes only what it cares about
public sealed class OrderBuilder
{
    private string country = "TR";
    private int completedOrders;
    private readonly List<OrderLine> lines = [];

    public static OrderBuilder AnOrder() => new();

    public OrderBuilder ShippedTo(string countryCode)
    {
        country = countryCode;
        return this;
    }

    public OrderBuilder ForLoyalCustomer()
    {
        completedOrders = 7;
        return this;
    }

    public OrderBuilder WithBook(decimal price, int quantity = 1)
    {
        lines.Add(new OrderLine(new Book($"978-{lines.Count:D10}", "Any Book", price), quantity));
        return this;
    }

    public Order Build()
    {
        Customer customer = new("C-1", "Ada", "ada@example.com",
            new Address("1 Main St", "Istanbul", "34000", country), completedOrders);
        IReadOnlyList<OrderLine> orderLines = lines.Count > 0 ? lines : [new OrderLine(new Book("978-0000000000", "Any Book", 10m), 1)];
        return new Order(Guid.NewGuid(), customer, orderLines);
    }
}

public sealed class ShippingPolicyTests
{
    [Fact]
    public void Domestic_order_below_50_pays_4_99() =>
        Assert.Equal(4.99m, ShippingPolicy.FeeFor(AnOrder().WithBook(49.99m).Build()));

    [Fact]
    public void Domestic_order_of_50_ships_free() =>
        Assert.Equal(0m, ShippingPolicy.FeeFor(AnOrder().WithBook(25m, quantity: 2).Build()));

    [Fact]
    public void International_order_pays_12_regardless_of_amount() =>
        Assert.Equal(12.00m, ShippingPolicy.FeeFor(AnOrder().ShippedTo("DE").WithBook(200m).Build()));
}

Each test is one line and states only the two things the shipping rule looks at: country and amount. The customer's name, email and street address are hidden in the builder's defaults; the test tells the reader "these do not matter". When a new required field is added to Customer tomorrow, only the builder changes, not the tests. This is the scalable version of the "minimal input" rule from post 03.

A good builder has two properties. Its defaults produce a valid object: AnOrder().Build() works on its own, including a default line. Its method names come from the domain language: ForLoyalCustomer() says more than WithCompletedOrders(7); the builder knows how many orders loyalty starts at, and if the rule changes it is fixed in one place.

Object Mother and the Growth Problem

The Object Mother is an older idea: a factory class that produces ready-made, named examples. Fowler's bliki mentions its use at ThoughtWorks:

// Object Mother: ready-made, named examples
public static class TestOrders
{
    public static Order SmallDomesticOrder() =>
        OrderBuilder.AnOrder().WithBook(20m).Build();

    public static Order LargeDomesticOrderFromLoyalCustomer() =>
        OrderBuilder.AnOrder().ForLoyalCustomer().WithBook(60m).Build();

    public static Order SmallInternationalOrderFromLoyalCustomer() =>
        OrderBuilder.AnOrder().ShippedTo("DE").ForLoyalCustomer().WithBook(20m).Build();
}

The three methods read well, and TestOrders.SmallDomesticOrder() states the intent nicely in a test. The trouble starts as it grows: three dimensions like country (2), loyalty (2) and amount (2) mean eight combinations and eight methods. One test wants "small, domestic, loyal customer, two lines", and a new method is added; another test wants to tweak the data of an existing method a little, and the other tests using that method are silently affected. Pryce proposed the Test Data Builder exactly for this problem. A practical balance: use the Object Mother for a few canonical examples that the team talks about a lot, and build it on top of the builder (as above); let tests make their variations themselves with the builder.

AutoFixture: Anonymous Data

Mark Seemann's AutoFixture answers a different question: if a test does not care about a value at all, why should I write it? It calls constructors through reflection and fills every argument:

[Fact]
public void AutoFixture_fills_every_constructor_argument()
{
    Fixture fixture = new();

    Customer customer = fixture.Create<Customer>();

    Assert.StartsWith("Email", customer.Email);
    Assert.StartsWith("Country", customer.ShippingAddress.Country);
}

The customer it produces looks like this (two runs gave two different values):

AF: Customer { Id = Iddeb9b579-b338-4aa7-86f0-89c67948ba68, Name = Name01f4ab89-c81a-44db-9f7b-faeac6e9d12a, Email = Emailba9a5c7f-1d16-4905-b315-13e7dd2610af, ShippingAddress = Address { Street = Street07d69422-5754-4555-af5c-058feeb7b9d2, City = City15523257-7b57-4cf5-9568-aa98d015d6f6, PostalCode = PostalCode6c7d0df8-5250-495f-932d-51ebad661923, Country = Countryd9590a1b-55dd-4f0c-852a-e758735f0989 }, CompletedOrders = 43 }

Strings are the parameter name plus a Guid, numbers are random values that differ on each run (43 in this run, 247 in the next). This "anonymous" data is deliberately unrealistic: because Country is not "TR", a shipping test written with this customer always falls into the international rule. AutoFixture shines in tests where the value truly does not matter. With the xUnit integration, parameters arrive directly in the test method:

[Theory, AutoData]
public void Order_subtotal_sums_line_totals(Customer customer, Book book, Guid id)
{
    Order order = new(id, customer, [new OrderLine(book, 3)]);

    Assert.Equal(book.Price * 3, order.Subtotal);
}

AutoFixture.AutoNSubstitute fills interfaces automatically with NSubstitute fakes; with Freeze you can hand the same instance to both the test and the code under test. Watch the package versions: AutoFixture's latest stable version is 4.18.1 (5.0 is at the RC (Release Candidate) stage), and AutoFixture.AutoNSubstitute 4.18.1 limits NSubstitute to < 6.0.0. With NSubstitute 6.2.0, restore gives this warning:

warning NU1608: Detected package version outside of dependency constraint: AutoFixture.AutoNSubstitute 4.18.1 requires NSubstitute (>= 2.0.3 && < 6.0.0) but version NSubstitute 6.2.0 was resolved.

In the scenarios I tried (Freeze and Received) it worked, but this is not a supported combination. AutoFixture.Xunit3 4.19.0, on the other hand, depends on xunit.v3.extensibility.core 1.0.0, and [AutoData] worked without problems with xUnit 4.0.2. Version limits like these are another face of the dependency lesson from post 07: test tools have maintenance schedules too.

Bogus: Realistic Data and a Fixed Seed

Bogus produces realistic-looking data: names, emails, addresses, product names, EAN (European Article Number) barcodes. It is valuable for UI (User Interface) screenshots, demo databases, performance experiments with large data, and tests where the shape of real data matters, such as sorting and search. There are two rules to avoid repeating the mistake from Predict First: do not leave the value a test relies on to chance, and pin the randomness with a seed:

[Fact]
public void Bogus_with_a_fixed_seed_is_deterministic()
{
    Faker<Customer> customers = new Faker<Customer>()
        .UseSeed(2026)
        .CustomInstantiator(f => new Customer(
            f.Random.Guid().ToString("N")[..8],
            f.Name.FullName(),
            f.Internet.Email(),
            new Address(f.Address.StreetAddress(), f.Address.City(), f.Address.ZipCode(), "TR"),
            f.Random.Int(0, 20)));

    Customer first = customers.Generate();
    TestContext.Current.TestOutputHelper?.WriteLine($"{first.Name} <{first.Email}> {first.CompletedOrders}");

    Assert.Equal("TR", first.ShippingAddress.Country);
}

With the same seed, the first customer was the same in two separate runs: "John Bruen", "Pauline.Anderson80@yahoo.com", city "Blockmouth", 6 orders. Notice that the name and the email were generated independently; if they need to be consistent, you have to tie them together with overloads like f.Internet.Email(firstName, lastName). A seed makes a red test reproducible; writing the value into the failure message or the test output makes the cause of the red readable. The rule stays the same: the country is fixed to "TR", because the test relies on it.

DRY or DAMP?

In production code, repetition is a smell; the DRY (Don't Repeat Yourself) principle says to gather repeated knowledge in one place. In test code the balance is different. The unit testing chapter of Software Engineering at Google (2020) recommends DAMP (Descriptive And Meaningful Phrases) for tests: a little repetition is an acceptable price for a test that can be read on its own. If understanding a test requires jumping to three helper methods, a base class and a fixture, repetition has been reduced but readability has been lost.

A practical line: the mechanics of building objects (constructor arguments, required fields) go into the builder, because they are not what the test wants to say. The values the test wants to say (49.99 EUR, "DE") stay in the test, even if they repeat. The same question applies to shared assertions: a helper like AssertIsPaidOrder(order) that groups a few asserts under a meaningful name is good; one written only to save lines that hides the failure message is bad.

In Production

Keep builders in a common folder of the test project and update them when the domain changes. If a builder's default starts producing an invalid object (because of a new validation rule), the hundreds of tests that use it turn red at once. This is not a bad thing: you fix one place. In a suite without builders, the same change meant fixing hundreds of tests one by one.

If you use random data, make the seed visible. Putting the seed in as a fixed value is the simplest way. If you want a different seed on every run, write the seed to the test output so a red run can be reproduced on a local machine with the same seed.

Watch dependency constraints. Instead of silencing the NU1608 warning with NoWarn, follow the new version of the package concerned, or accept the combination deliberately and explain why in a comment in the project file.

Common Pitfalls

A test that breaks now and then because of random data. Predict First itself. The condition in the test name (below the threshold) must be guaranteed by the data. Randomness should stay in the fields the test does not care about.

Repeating business rules inside the builder. If a builder method like WithDiscountedTotal() computes the discount itself, the tests run against a copy of the production discount rule; it is the builder version of the "computing the expected value with production code" mistake from post 03. A builder sets up input; it does not compute results.

Giving meaning to anonymous data. AutoFixture's CompletedOrders = 43 is above the loyalty threshold; if a test relies on that, the test's behavior changes when AutoFixture's number generation strategy changes or when the order relative to other tests changes. Give the value you care about explicitly.

Growing the Object Mother. By the time the fortieth TestOrders method is added, no test knows which data the others rely on. Leave variations to the builder.

A Small Experiment

★☆☆ Fix the test from Predict First: keep the price random but always below the threshold (faker.Random.Decimal(5m, 49.99m)). Run the test 30 times and write the price to the test output.

★★☆ Add WithCoupon(string code) to OrderBuilder and decide what the default should be when no coupon code is given. Which of the existing tests had to change?

★★★ Pick the ten tests with the longest Arrange sections in your own project. For each, mark which values are what the test wants to say and which are there only to satisfy the constructor. When you move the second group into a builder, how many lines does the Arrange shrink to?

In the next post we get into the hardest corners of unit testing: async code, time, exceptions, logging and background services; advancing timers with FakeTimeProvider.Advance and testing a Channel<T> consumer.