Test Pyramid, Testing Trophy and Honeycomb: Which Shape Fits Which System?

Version note: The code in this post was built and run with .NET 10 (SDK 10.0.402), Entity Framework Core 10.0.14, xUnit.net v3 4.0.2, Angular 22.2 and TypeScript 6.0.

Guess First

We are adding book search to the Bookstore: GET /api/books/search?q=clean should return the books whose title contains "clean", ignoring case. With Entity Framework Core (EF Core) the query is a single line. We write two tests for it. The first calls the service directly with EF Core's InMemory provider and runs in under half a second. The second verifies the same behavior over HTTP (Hypertext Transfer Protocol) against real SQLite. One of them will go red. Which one? And what does the answer say about whether a thin CRUD (Create, Read, Update, Delete) API (Application Programming Interface) like this should follow the test pyramid or some other shape?

A Short History: From the Pyramid to the Honeycomb

The test pyramid became widely known through the 2009 book Succeeding with Agile by Mike Cohn (Scrum trainer, author of User Stories Applied). According to a short 2012 piece by Martin Fowler (chief scientist at Thoughtworks), Cohn had drawn it a few years earlier in a conversation with Lisa Crispin. The idea is simple: many fast unit tests at the bottom, fewer service-level tests in the middle, very few tests through the UI (User Interface) at the top. Fowler's reasoning is just as clear: UI tests are brittle, slow and expensive to write.

In 2018 "The Practical Test Pyramid" by Ham Vocke (a Thoughtworks consultant) boiled the shape down to two rules: write tests with different granularity; the higher you go, the fewer tests you should have. Vocke also admits that the layer names are fuzzy and suggests that a team agree on its own names.

The same year, Spotify engineers André Schaffer and Rickard Dybeck proposed the honeycomb for microservices. A few "implementation detail" tests at the bottom, integration tests as the largest layer in the middle, and at the top "integrated" tests, ideally none at all. In Spotify's definition, an integrated test is one that passes or fails based on the correctness of another system.

Kent C. Dodds (JavaScript educator, author of Testing Library) drew the testing trophy. Its starting point was a December 2016 tweet by Guillermo Rauch (founder of Vercel): "Write tests. Not too many. Mostly integration." The trophy has four layers: static analysis (type checking, linters) at the very bottom, then unit, integration as the widest layer, and a few E2E (End-to-End) tests on top. Dodds sharpened these definitions in his 2021 post "The Testing Trophy and Testing Classifications".

There is also a shape nobody wants: the ice-cream cone. Most tests go through the UI or are done by hand, and there are almost no unit tests; Fowler describes it as the pyramid turned upside down. Software Engineering at Google (Titus Winters, Tom Manshreck and Hyrum Wright, O'Reilly, 2020) adds the hourglass: many unit tests, many E2E tests, almost no integration tests in between. According to the book, it shows up in tightly coupled systems where parts are hard to separate from each other.

The Common Idea: Confidence Divided by Cost

The four shapes are presented as rivals, but they answer the same question: how much confidence do you get for every minute and every hour of maintenance you spend? Dodds describes it openly as a return on investment; the return is confidence, the investment is time. His guiding principle: the more your tests resemble the way your software is used, the more confidence they give.

What separates the shapes is at which level that ratio is highest in your system. The ratio depends on three things. Where the logic lives: is the complex computation in the domain, or in database queries and framework configuration? The cost of a test at each level: in the Bookstore a small unit test took 0–6 milliseconds, an integration test half a second, and an E2E test's environment four seconds. And where bugs show up: when you trace production incidents back, in which layer is the root cause usually found?

That is why "which shape is right?" is the wrong question. The right question is "in this system, which level gives confidence most cheaply?" Below we answer it for four different parts of the Bookstore.

A Domain-Heavy .NET API: Close to the Pyramid

The discount rule is a pure computation: a threshold, a customer tier and the combination of the two. In a rule like this, bugs hide at the boundaries, and the cheapest way to walk the boundaries is a unit test:

[Trait("Size", "Small")]
public class DiscountRateTests
{
    [Theory]
    [InlineData(0, CustomerTier.Standard, 0)]
    [InlineData(499.99, CustomerTier.Standard, 0)]
    [InlineData(500, CustomerTier.Standard, 0.10)]
    [InlineData(500.01, CustomerTier.Standard, 0.10)]
    [InlineData(0, CustomerTier.Gold, 0.05)]
    [InlineData(499.99, CustomerTier.Gold, 0.05)]
    [InlineData(500, CustomerTier.Gold, 0.15)]
    [InlineData(12000, CustomerTier.Gold, 0.15)]
    public void Rate_depends_on_threshold_and_tier(decimal subtotal, CustomerTier tier, decimal expected) =>
        Assert.Equal(expected, DiscountPolicy.RateFor(subtotal, tier));
}

=== TEST EXECUTION SUMMARY ===
   Bookstore.Domain.Tests  Total: 8, Errors: 0, Failed: 0, Skipped: 0, Not Run: 0, Time: 0.051s

Eight boundary values in 51 milliseconds. Had you tried to verify the same eight combinations through the browser, with the 0.4–0.5 seconds per test and four seconds of environment we measured in the previous post, you would spend roughly 8 seconds; and you would have to invent fake books just to build a 499.99 TL cart. If most of the logic is in the domain, meaning pricing, tax, campaign rules or booking algorithms, the pyramid really is the shape with the best ratio. A thin integration layer shows that these rules are wired correctly to the endpoint, and a few E2E tests show that the main flow works. We look in detail at what should be a unit test and what an integration test at this level in the next post.

A Thin CRUD API with EF Core: Integration-Heavy

Back to the search endpoint from Guess First. The service's whole job is one query:

public sealed class BookSearch(BookstoreDb db)
{
    public Task<List<Book>> SearchAsync(string term, CancellationToken ct) =>
        db.Books
          .Where(b => b.Title.Contains(term, StringComparison.OrdinalIgnoreCase))
          .OrderBy(b => b.Title)
          .ToListAsync(ct);
}

Following the pyramid reflex, we write a unit test first. Since a real database is "slow," we use EF Core's InMemory provider:

public class BookSearchTests
{
    [Fact]
    public async Task Finds_books_case_insensitively()
    {
        DbContextOptions<BookstoreDb> options = new DbContextOptionsBuilder<BookstoreDb>()
            .UseInMemoryDatabase(nameof(Finds_books_case_insensitively))
            .Options;
        await using BookstoreDb db = new(options);
        BookstoreDb.Seed(db);

        List<Book> books = await new BookSearch(db).SearchAsync("clean", TestContext.Current.CancellationToken);

        Assert.Equal(["Clean Architecture"], books.Select(b => b.Title));
    }
}

Next to it we add an integration test that verifies the same behavior through the API against real SQLite:

[Trait("Size", "Medium")]
public class BookSearchApiTests
{
    [Fact]
    public async Task Finds_books_case_insensitively()
    {
        await using BookstoreApiFactory factory = new();
        HttpClient client = factory.CreateClient();

        List<Book>? books = await client.GetFromJsonAsync<List<Book>>("/api/books/search?q=clean",
            TestContext.Current.CancellationToken);

        Assert.Equal(["Clean Architecture"], books!.Select(b => b.Title));
    }
}

tests/Bookstore.Api.UnitTests/bin/Debug/net10.0/Bookstore.Api.UnitTests.dll (net10.0|arm64) passed (334ms)
failed Bookstore.Api.Tests.BookSearchApiTests.Finds_books_case_insensitively (492ms)
  from tests/Bookstore.Api.Tests/bin/Debug/net10.0/Bookstore.Api.Tests.dll (net10.0|arm64)
  Xunit.MicrosoftTestingPlatform.XunitException: System.Net.Http.HttpRequestException : Response status code does not indicate success: 500 (Internal Server Error).

The unit test is green, the integration test is red. The actual error in the server log:

System.InvalidOperationException: The LINQ expression 'DbSet<Book>()
    .Where(b => b.Title.Contains(
        value: @term,
        comparisonType: OrdinalIgnoreCase))' could not be translated. Additional information: Translation of method 'string.Contains' failed.

The InMemory provider does not translate the query to SQL (Structured Query Language); it runs it in memory, like LINQ (Language Integrated Query) to Objects, where Contains with a StringComparison argument works fine. The SQLite provider cannot translate that overload, and in production the endpoint would have returned 500 on every request. The unit test removed from the test exactly the place where the code can break: the translation of the query to the database. The EF Core documentation also advises against testing with the InMemory provider; the EF Core series post "Testing with EF Core: SQLite In-Memory and InMemory Provider Pitfalls" explains why in detail.

The fix is one line, and it is the integration test that tells us it is correct:

.Where(b => EF.Functions.Like(b.Title, $"%{term}%"))

Test run summary: Passed!
  tests/Bookstore.Api.Tests/bin/Debug/net10.0/Bookstore.Api.Tests.dll (net10.0|arm64) passed (719ms)
  tests/Bookstore.Domain.Tests/bin/Debug/net10.0/Bookstore.Domain.Tests.dll (net10.0|arm64) passed (181ms)
  tests/Bookstore.Api.UnitTests/bin/Debug/net10.0/Bookstore.Api.UnitTests.dll (net10.0|arm64) passed (421ms)

In SQLite, LIKE is case-insensitive for ASCII (American Standard Code for Information Interchange) characters. In SQL Server the result depends on the column's collation; again, a detail you can only verify with a test that runs against a real database. In an API where most of the logic lives in queries, mappings, validation attributes and middleware order, integration tests give confidence most cheaply. Unit tests are left for the small pieces that actually compute something. This is a trophy, or a honeycomb drawn for a single service.

An Angular SPA: The Trophy and the Static Layer

The bottom layer of the trophy is something most .NET developers don't count as "testing": static analysis. Dodds says he added it because JavaScript does not have type checking by default. In an Angular project the TypeScript compiler is that layer. When a new function that adds a gift book to the cart writes the price as a string:

export function addGiftBook(cart: CartStore): void {
  cart.add({ id: 9, title: 'Refactoring', price: '310', stock: 3 });
}

$ time npx tsc -p tsconfig.app.json --noEmit
src/app/gift-card.ts(4,43): error TS2322: Type 'string' is not assignable to type 'number'.
npx tsc -p tsconfig.app.json --noEmit  0.79s user 0.17s system 119% cpu 0.804 total

Without writing a single test, in 0.8 seconds. A unit test would have caught the same bug, but only after someone thought of writing it. On the C# side, a project with Nullable enabled plus analyzers does the same job.

In an SPA (Single Page Application) most of the behavior lives in how components are wired to each other, to services and to templates. The test from the previous post that renders the cart component together with its store, template and pipe is an integration test by Dodds' definition; tests like that make up the widest layer of the trophy. Small functions that do pure computation get unit tests, and a few critical flows (login, cart, payment) get E2E tests with Playwright. The tools on the Angular side are the subject of the upcoming Frontend Testing series.

Microservices: The Honeycomb and Contracts

When the Bookstore grows and orders, stock and shipping are split into separate services, the picture changes again. The observation in Spotify's article: because a microservice is small, the real complexity is not inside it but in how services talk to each other. Writing unit tests for every class ties the tests to the implementation; every change brings a test change.

The integration test in the middle of the honeycomb starts the service with its own database and talks to it only through its externally visible API. For the order service this means putting test doubles in place of the stock and shipping services and testing its own HTTP endpoints and its own database for real. Contract tests guarantee that those test doubles stay compatible with the real services: the shipping team learns in its own pipeline the moment it breaks fields the order service expects. The mechanics of consumer-driven contracts are covered in the Microservices series post "Service Contracts and Versioning: Expand-Contract, Tolerant Reader and Consumer-Driven Contract Tests".

The integrated tests at the top of the honeycomb, the ones that need every service running in a shared environment, are avoided. Spotify's warning signs are familiar: running other services locally to run a test, testing against a shared test environment, and your change breaking another team's tests.

In Production

Measure your real shape. Put the test count and total duration of each test project in a table and group them by level. When most teams do this for the first time, they see an hourglass or an ice-cream cone instead of the pyramid they thought they had drawn. The 80 percent unit, 15 percent integration, 5 percent E2E split in the Google book is a starting estimate; the book itself says every team's mix will differ.

Label production incidents by layer. Was the root cause of the last ten incidents in a domain computation, a query, configuration, or the contract between two services? That distribution tells you where to invest better than any diagram.

Separate the levels in the pipeline. Small tests on every commit, integration tests on the pull request, E2E and contract verification before the deploy. The details are in the Continuous Integration / Continuous Delivery (CI/CD) Fundamentals series post "Testing in the Pipeline: Fast Feedback, Quality Gates and Flaky Tests".

Common Pitfalls

Making the shape the goal. A team that raises an alarm because "our unit tests dropped below 70 percent" fixes the ratio by writing worthless unit tests like the one for the search endpoint, and loses confidence. The shape is a picture that emerges from good decisions.

Making E2E the only safety net. A suite that tests everything through the browser, as we saw in the previous post, trips over shared data, slows down and turns flaky. And when it goes red, it does not tell you which layer the bug is in.

Testing everything end to end in microservices. A staging environment where ten services must be up at the same time puts every deploy into a queue shared by all teams. That is the exact opposite of independent deployability, the main reason for moving to microservices.

Mistaking the InMemory provider for a "fast integration test." As shown above, a test that does not exercise query translation tests almost nothing in a thin CRUD API.

Forgetting the static layer. A codebase that silences Nullable warnings or is full of any has thrown away the cheapest layer of the trophy with its own hands.

A Small Experiment

★☆☆ Count the tests in your own project by level and write down the total duration of each level. Which of the four shapes does the result resemble?

★★☆ Classify the root causes of your last ten production bugs as domain, query, configuration, contract and UI. Which test level catches the largest class most cheaply?

★★★ Rewrite BookSearchTests with SQLite in-memory and bring back the query with StringComparison. Does the test go red? With a Turkish collation in SQL Server, does searching for "ISTANBUL" find the "istanbul" record, and at which test level would you verify that?

Choosing a shape means reading the character of the system. But even within a single feature you will have to decide line by line: is the discount calculation a unit test, is the stock deduction an integration test, is the payment service mocked or real? In the next post we make those decisions with the code quadrant of Vladimir Khorikov (author of Unit Testing Principles, Practices, and Patterns) and the managed/unmanaged dependency distinction.