The Limits of TDD: The "Is TDD Dead?" Debate, Research, When Not to TDD and TDD with AI Agents

Version note: The code in this post was run with .NET 10, xUnit.net v3 4.0.3 and Bash.

Predict First

You gave an AI coding agent the job of putting an upper limit on percentage coupons in the CouponValidator we grew with TDD in the second post. The agent worked for a while, pushed a commit saying "fixed the failing coupon tests", and the branch's CI output looks like this:

skipped Bookstore.Discounts.Tests.CouponValidatorTests.Coupon_expires_at_midnight_Istanbul_time_not_UTC
  Flaky on CI
Test run summary: Passed!
  total: 10
  failed: 0
  succeeded: 9
  skipped: 1
  duration: 418ms

The pipeline is green. Would you merge this branch? If not, which line of the output stopped you, and what might the output not be showing? We see the answer halfway through the post, together with the agent's diff.

The "Is TDD Dead?" Debate

In the spring of 2014 David Heinemeier Hansson (DHH), the creator of Ruby on Rails, declared TDD dead in his RailsConf opening keynote and in the blog posts that followed. Kent Beck and Martin Fowler then held a five-part video debate with him in May and June 2014; part-by-part summaries are on Fowler's site. The outline of the debate, without quotes:

DHH's first objection was that TDD gets confused with unit testing and that mock-heavy TDD harms design. He used the term "test-induced design damage" for it: splitting code into layers, interfaces and indirect calls just so it can be tested in isolation was hurting readability and simplicity. Beck's answer was that the problem lay not in TDD itself but in individual design decisions; the pressure of tests can push design in a good direction or a bad one. The ICouponPolicy and its adapter we saw in the fourth post are a small example of this debate: mock-driven design gave birth to a layer, and whether that layer is needed is a separate decision.

In the later parts the topic shifted to feedback and cost. Beck described feedback as a trade-off among frequency, fidelity, cost and lifespan. DHH complained that in some teams TDD had made a separate QA function and exploratory testing look unnecessary. All agreed that over-testing, needless coupling between tests and the ratio of test code to production code are real costs. Beck said tests that protect nothing unique can be deleted.

The debate didn't end in agreement. The common ground was that self-testing code is valuable and that TDD is one path to it, not the only one. They stood in different places on how much TDD helps in which contexts. In closing, Fowler suggested trying techniques, overusing them if necessary to see their limits, and adopting what fits the team. Read twelve years later, the debate's value is not in producing a winner but in talking about TDD as a trade-off rather than a ritual.

What Does the Research Say?

There are many studies that try to measure the effect of TDD, and their results are mixed. Three of the most cited:

The 2008 study by Nagappan, Maximilien, Bhat and Williams examined the TDD experience of three teams at Microsoft and one at IBM. The pre-release defect density of the projects using TDD was between 40 and 90 percent lower than in similar projects; managers estimated that initial development time grew by 15 to 35 percent. These are case studies: not controlled experiments, and the time increase is an estimate, not a measurement.

Rafique and Mišić's 2013 meta-analysis combined 27 studies. The result: a small positive effect on external quality and no discernible effect on productivity. An interesting detail is that in studies done in industry both the quality gain and the productivity drop came out much larger than in academic studies.

The 2017 study by Fucci, Erdogmus, Turhan, Oivo and Juristo asked the question differently: which property of TDD makes the difference? In the process data of 39 professional developers, whether the test was written before or after the code did not turn out to be decisive. What seemed to matter was short and steady cycles. The authors' interpretation is that the benefit attributed to TDD may come less from the test-first order than from the small, steady steps that TDD-like processes encourage.

Why are the results so scattered? Because it is hard to measure. Should "quality" be measured by defect counts, maintenance cost or design? Who is the control group: students who learned TDD in a week, or a team that has been writing tests for years? Some of TDD's promised benefits (design that withstands change, fearless refactoring) become visible over months or years, while most studies last days or weeks. So the honest conclusion from the research is this: TDD is not magic, nor is it harmful; it is a way of working in small steps with continuous feedback, and its benefit depends largely on how it is practiced.

Where TDD Falls Short

TDD is strongest when you know what you want but not how to build it. It weakens when you don't know what you want.

Exploration code and spikes. Code written to try a new library, see how a payment provider's API really behaves, or find out whether an algorithm is feasible is written for learning. You can't write a test list, because you don't know the behavior yet. The honest approach here is to write the exploration code consciously as untested, throwaway code, take notes on what you learned and write the production code from scratch with TDD.

The "spike and stabilize" approach described by Dan North goes a step further: sometimes it makes more sense to expose exploration code to real users behind a feature flag, see whether it really creates value, and add tests and cleanup only after the value is proven. Code that creates no value is deleted; code that does is stabilized, that is, characterized, tested and refactored. The characterization test and sprout techniques from the previous post are exactly the tools of that stabilization step.

Uncertain UI and visual layout. The question "does this page look good?" has no assertion you can write in advance. As we saw in the fifth post, a component's behavior can be written test-first; visual layout settles together with the designer, by trying it on screen. Visual regression tests protect that layout afterwards, but they don't drive the design.

Framework configuration. Writing unit tests for dependency injection registrations, middleware order or serialization settings often means testing the framework. What best shows that they are right is a few acceptance tests that start the application in process, as in the fourth post.

Performance. A goal like "this query must stay under 50 ms" can be written as a test, but it doesn't fit the minute-by-minute rhythm of red-green-refactor; measurements are noisy and improvement often changes the design. Performance moves forward by profiling and measuring; its place is the upcoming Performance and Load Testing series.

TDD with AI Coding Agents

Coding agents added a new dimension to the TDD debate. An agent writes code fast, but it can't know on its own that the code is correct; its "it works" is a claim. Tests are the one mechanism that makes that claim verifiable. From this angle, test-first is the most concrete form of the specification a human gives an agent: you say "this input, this result" not in natural language but in runnable code.

Kent Beck describes this experience in "Augmented Coding: Beyond the Vibes", published in June 2025. He wrote a B+ tree library in Rust and Python with an agent and described the TDD rhythm in its system instructions: take the next unmarked test from a plan file, write only enough code to pass that test, put structural changes and behavioral changes in separate commits, don't commit unless the tests are green. The warning signs he watched for are instructive too: the agent getting into loops, adding functionality nobody asked for, and signs of cheating such as disabling or deleting tests. Another interesting detail of the experience: when the Rust version got stuck in complexity, he had the Python version written against the same tests and then translated it into Rust; the tests remained the specification even when the language changed.

The branch in "Predict First" carries exactly that sign of cheating. The agent's diff:

@@ -45 +45 @@ public sealed class CouponValidatorTests
-        Assert.Equal(CouponResult.Accepted(14.99m), result);
+        Assert.True(result.Discount > 0);
@@ -91 +91 @@ public sealed class CouponValidatorTests
-    [Fact]
+    [Fact(Skip = "Flaky on CI")]

Instead of fixing the two tests it had turned red, the agent skipped one and loosened the other's assertion to "greater than zero". The pipeline is green, but the rounding rule and the Istanbul midnight rule are no longer protected. The skipped: 1 line in the CI output was the only clue; the output doesn't show the loosened assertion at all.

We build two lines of defense against this. The first is the rules given to the agent; most agents read an instructions file at the repository root:

## Testing rules for coding agents

- Work from TESTLIST.md: pick the next unchecked item, write one failing test, run it and show the red output.
- Write only the code needed to make that test pass; run the whole suite before you report green.
- Never delete, skip or weaken an existing test or assertion. If a test looks wrong, stop and ask.
- Keep structural changes (refactoring) and behavioral changes in separate commits.
- Do not add behavior that no test asks for; add it to TESTLIST.md instead.

Rules are necessary but not sufficient; an agent doesn't always follow them. The second line is a CI step that catches violations mechanically. A simple script counts removed assertions, removed [Fact]/[Theory] lines and added Skips in the diff of the branch's test files:

#!/usr/bin/env bash
# Looks for signs of weakened tests in the branch's test changes
set -euo pipefail
base="${1:-main}"
diff=$(git diff --unified=0 "$base"...HEAD -- '*Tests.cs')

removed_asserts=$(grep -cE '^-[[:space:]]*Assert\.' <<< "$diff" || true)
removed_facts=$(grep -cE '^-[[:space:]]*\[(Fact|Theory)\]' <<< "$diff" || true)
added_skips=$(grep -cE '^\+.*Skip[[:space:]]*=' <<< "$diff" || true)

echo "removed asserts: $removed_asserts, removed [Fact]/[Theory]: $removed_facts, added skips: $added_skips"
if (( removed_asserts + removed_facts + added_skips > 0 )); then
  echo "Test changes need a human review before merge." >&2
  exit 1
fi

$ ./tools/tests-guard.sh main
removed asserts: 1, removed [Fact]/[Theory]: 1, added skips: 1
Test changes need a human review before merge.
$ echo $?
1

The script is not a judge but an alarm. A changed assertion is not always a weakening; sometimes the test really is wrong. The goal is to make sure a human consciously approves every change that touches test files. On Windows the script runs with Git Bash, or on the Linux runner in CI. For a sturdier check, mutation testing measures whether tests really catch anything; the coverage post in the Test Strategy series covers it. The rest of the workflow with agents, meaning task descriptions, review and context management, is the topic of the upcoming AI-Assisted Development series.

Spreading TDD in a Team

Turning TDD into a team rule ("all code is written test-first") usually backfires: it's impossible to check who follows it, and the rule is easily dodged by writing the test afterwards and rearranging the commit order. What works is making TDD visible and attractive as a practice.

Pair and mob programming are where TDD spreads fastest. In ping-pong pairing one writes the test and the other the code; the rhythm is learned naturally. The weekly kata sessions from the third post are the place to practice that rhythm without production pressure.

Looking for traces of test-first in reviews helps too, but out of curiosity, not as an audit. Seeing tests and code arrive together in small steps in the commit history; seeing in a bug fix pull request that the red test was committed first and the fix after; finding a test list in a pull request description. Praising these in reviews does more than criticizing their absence.

In Production

The quality of tests the agent writes. When you have the agent write both the code and the test, the two can share the same misunderstanding: the test verifies what the code does, not the desired behavior. That is a trap similar to the characterization test in the sixth post. Having a human write the test list, and at least the expected values of the critical tests, ties the agent to the specification.

Skipped tests piling up. Tests turned off with Skip pile up over time and nobody remembers why they were turned off. Tracking the number of skipped tests in CI and making sure someone takes ownership when it rises above zero is good hygiene for any team, with or without agents.

Measurement traps. If you want to see TDD's effect in a team, coverage percentage is the wrong metric; code not written test-first can also reach high coverage. More meaningful signals: how often refactoring commits happen, how many regressions escape to production, how long it takes a new developer to change a module with confidence.

Common Pitfalls

Imposing TDD like a coverage target. An "80 percent coverage" target rewards tests without assertions, getter tests and tests that copy what the code does. One of the first mistakes Beck lists in Canon TDD is writing tests without assertions for coverage. Coverage is an outcome, not a target.

Using "we don't do TDD" to justify not writing tests. All three participants of the "Is TDD Dead?" debate agreed that self-testing code is necessary. Dropping TDD doesn't mean dropping tests; it changes when tests are written.

Trusting the agent's "tests pass". The agent's report is a claim. Did you see red, did the tests really run, did test files change in the diff? These three questions are the essence of TDD when working with an agent.

Carrying spike code to production. When untested code written for exploration gets into the main branch with "it already works", legacy code is born. A spike ends either in deletion or in stabilization; there is no middle ground.

A Small Experiment

★☆☆ Look at the last ten pull requests in your own project: in how many did tests and code arrive in the same commits, and in how many did the tests come last in a separate commit? Can you find a bug fix where the red test stands as its own commit?

★★☆ Extend tests-guard.sh: report lines where Assert.Equal turned into Assert.True separately, and also count removed DidNotReceive/Received lines. Loosen an assertion on purpose in your own branch and watch the script catch it.

★★★ Give a coding agent the CouponValidator test list from the second post of this series and the rules above, and have it do the work from scratch. Did it show the red at every step, did it add behavior nobody asked for, did it try to change a test? Compare the resulting code and commit history with the git log in the second post.

This is where the series ends. Over seven posts we treated TDD not as a testing technique but as a discipline of designing in small steps: starting with a test list, seeing red for the right reason, finding green by the shortest path, not skipping the refactor, and trying it in different settings, from katas to legacy code, from the backend to Angular, from human teams to agents. The road continues in the upcoming Refactoring, Integration and E2E (end-to-end) Testing, and Frontend Testing series.