Testing in the AI Era — What Changed, and What Question Didn't

Large language models (LLMs) have found their way into code generation, and from there into generating, auto-repairing, and evaluating test code as well. Tools like GitHub Copilot, Claude (including Claude Code), and Cursor have seen growing adoption in practice. At the same time, academic research and field reports in this space have revealed about as many limitations as they have promises. The essential difficulty of test automation is the oracle problem — who decides what counts as a correct result, and how — and this problem predates LLMs entirely; LLMs haven’t resolved it. As the final article in this series, we lay out, without exaggeration, what’s actually known about the state of the art and its limits.

The Reality of LLM-Generated Tests

Coding assistants like Copilot, Claude, and Cursor can read a target function, its comments, and its type information as context, and generate the skeleton of a test case — representative input values, the shape of the assertions. Practical value has been confirmed in a few specific situations: quickly drafting boilerplate tests that resemble existing patterns, recalling edge cases developers tend to overlook (empty strings, null, boundary values), and generating a first-pass “test that records current behavior” for legacy code that has none.

Academic evaluation studies, on the other hand, have repeatedly reported problems: generated tests that contain compile errors, that settle for trivial, low-value assertions, or that fail to account for repository-specific naming conventions and helper functions. Multiple studies also converge on the same finding: generation accuracy depends heavily on how much context (type signatures, examples of existing tests, code comments) is given about the target code.

The Hype and Reality Behind “Self-Healing”

Self-healing is one of the terms in AI-era testing with the widest gap between hype and reality — though some of it has genuinely reached production use. Commercial E2E test automation tools like Mabl, Testim, and Functionize offer an auto-healing capability: when a “locator” identifying a UI element stops working because the DOM structure changed, the tool matches multiple signals — the element’s attributes, its visual position, the surrounding DOM structure — with a machine learning model, and selects the best-matching replacement element. This is a limited but genuinely productionized answer to the structural fragility of E2E tests covered in the earlier article The Fight Against Flaky Tests.

A broader notion of self-healing — “when a test fails, AI autonomously rewrites the test code itself to fix it” — remains squarely in the research stage. What commercial tools actually ship is, for the most part, limited to something like automatic locator adjustment.

The Oracle Problem — A Fundamental Question AI Hasn’t Solved

The test oracle problem is the fundamental question in test automation of how to decide whether a program’s output is correct. It’s a non-issue when a simple equality check on input/output suffices, but it becomes serious when “the correct output” is itself hard to define (natural language generation, image recognition, recommendation systems), when the cost of preparing expected output by hand is prohibitive, or when multiple valid outputs can exist.

LLMs are useful for suggesting candidate test inputs (boundary values, abnormal-case patterns), but they are not a substitute for the authority — the oracle — that decides what output is correct. You can ask an LLM “is this output correct?”, but that just delegates the judgment to another LLM call; it provides no grounding for the correctness of the judgment itself. The oracle problem was a central unsolved issue before LLMs existed, and AI hasn’t resolved it — it’s more accurate to say the question of who makes the call, and how has simply taken on a new shape while persisting.

Domains where an oracle is easy to define	Domains where an oracle is hard
numerical computation, sorting, parsing — tasks solvable by equality checks	natural language generation, summarization, conversational responses
tasks with abundant known input/output examples	image recognition, recommendation systems, where “correct” is probabilistic or subjective
tasks where the expected value can be mechanically derived from a spec	tasks where multiple valid outputs can exist (an LLM returning the same meaning in different phrasings)

The former is the domain where classic unit testing works as-is; the latter is exactly the domain that constantly runs into trouble in evaluating LLM-powered systems themselves (the “eval” discussed below).

AI Inference of Properties and Invariants

Property-based testing, covered earlier in this series, verifies properties that should always hold, rather than individual input/output examples. There are research-level attempts to use LLMs to extract candidate invariants: having an LLM propose candidate properties from code or documentation — “this function’s result should always be larger than its input,” for instance — and letting a human decide whether to adopt them as preconditions or postconditions. Even here, it’s still a human who makes the final call on whether the property the LLM proposed is actually correct for the domain; the oracle problem isn’t sidestepped.

Testing Strategy for AI-Generated Code

On the question of how to test AI-generated code itself, the widely shared position is that “it’s no fundamentally different from testing human-written code.” What matters more in practice: pulling in a large volume of generated code without review means the intent has to be reverse-engineered from the tests alone, which makes test design itself harder. Generating the implementation and its tests from the same prompt and the same model call risks producing bugs in the implementation and errors in the test’s assertions that are correlated with each other, undermining independent verification. And even when AI proposes the implementation, it’s healthier — as a defense against the oracle problem — for a human to be the one who defines the expected behavior.

Testing AI Systems Themselves — Eval, LLM-as-Judge, and Non-Determinism

Testing a system that has an LLM embedded in it (a chatbot, an agent, a summarization or classification pipeline) is different in nature from testing a conventional, deterministic unit test. This space tends to use the word eval rather than “test.” LLM-as-judge is a technique where another LLM call judges the quality of an LLM’s output; it lowers the cost of human evaluation, but the judging LLM has its own biases and instability, so the reliability of the judgment itself needs to be verified.

Empirical research has shown that even with temperature set to 0, an LLM’s output doesn’t become fully deterministic — residual non-determinism from batching and hardware optimization in the inference stack remains. This breaks the conventional testing assumption that a single run can settle a pass/fail verdict. Practical responses that are gaining traction: running the same prompt multiple times (roughly 3–10) and reporting the mean and variance; verifying semantic equivalence rather than exact match (schema conformance, instruction-following, structural correctness); and recording the model version, sampling parameters, and evaluator version to preserve reproducibility. This means the concept of “non-determinism” covered in the earlier flaky-tests article shows up in LLM-containing systems not as an incidental bug, but as an inherent property of the system.

Risks — Hallucinated Tests, Over-Trust, the Illusion of Coverage, and Implementation-Confirming Tests

LLM-generated tests carry a few concrete risks that have been documented in the field. Hallucinated tests — generating tests that call nonexistent APIs, use the wrong type signature, or don’t even compile. Over-trust — the assumption that “the AI generated it, so it must be right,” which lets reviews get skipped. The illusion of coverage — mass-generating tests with raising the coverage number as the goal in itself, which raises the coverage percentage while piling up tests that carry no real bug-detection power.

The most serious of these is implementation-confirming tests. This is where an LLM, generating a test, builds its assertion from “what the current implementation actually returns” rather than from “what the correct behavior should be” — with particularly severe consequences for code that already contains a bug. One study evaluating three test-generation tools — GitHub Copilot, CodiumAI CoverAgent, and CoverUp — found that up to 68.1% of the resulting test suites passed against the buggy, incorrect implementation and, conversely, failed against the correct one — meaning the tests had effectively locked in the bug itself as the “correct” behavior. This is cited as a textbook case of a design choice aimed at raising coverage numbers producing bug-confirmation rather than bug-detection.

Human Review as an Indispensable Step

Given all this, the realistic position that’s broadly accepted at present looks like this. LLMs carry real practical value for generating test scaffolding, recalling candidate inputs, and speeding up routine work. LLMs cannot substitute for the authority that decides what the correct behavior is — not because of a technical limitation, but because the oracle problem itself fundamentally demands human domain knowledge and understanding of the specification. Generated tests should go through human review just like any other pull request, with particular attention to whether an assertion is merely tracing the current state of the implementation (an implementation-confirming test). The principle that a test should verify “behavior that conforms to the spec,” not “how the code currently behaves,” hasn’t changed in the AI era.

Sixty Years in the Real World, Eighteen Articles in This Series

Compressed down, the real-world history of software testing looks like this.

Era	In one line
through the 1970s	an extension of manual checking and debugging; “testing” wasn’t yet a discipline of its own
mid-1990s	SUnit establishes the archetypal xUnit patterns — fixtures, test runners
1997–2002	JUnit spreads and TDD is systematized; testing becomes something that drives design
late 2000s	BDD, Cucumber, and xUnit Test Patterns crystallize the vocabulary and techniques
early 2010s	JS/frontend frameworks (Jasmine → Jest) explode in popularity
late 2010s	Selenium’s dominance gives way to Cypress and Puppeteer, a generational shift in E2E tools
2020s	Playwright unifies multi-browser testing, and LLM-generated tests take hold

Looking Back Across the Series — the Question Running Through 18 Articles

This series split that timeline across 18 articles.

Article	In one line
1. Prehistory	the age of manual checking, Dijkstra’s aphorism, the classic V&V distinction
2. The Birth of xUnit	SUnit/JUnit invent the fixture and the test runner
3. The Spread of xUnit	xUnit-family patterns spread across language ecosystems
4. TDD	Red-Green-Refactor; testing drives design
5. BDD	lowering testing’s psychological barrier through the language of “behavior”
6. Mocks and Test Doubles	systematizing the technique of substituting real dependencies
7. Property-Based Testing	verifying properties that always hold, not individual examples
8. E2E Generations	from Selenium to Cypress to Playwright
9. JavaScript Testing	the explosive rise of frontend testing
10. Language Ecosystems	how testing culture differs across language communities
11. The Shapes of Tests	theories of distribution, like the test pyramid
12. Unit Test Design and Contracts	what makes a good unit test, design by contract
13. Coverage and Mutation	how to measure “we’ve tested enough,” and the limits of measuring it
14. Flaky Tests	non-determinism, the problem eating away at test reliability
15. Formal Methods	reaching for mathematical proof where testing can’t go
16. Quality Is Everyone’s Job	testing moves from a discipline to a concern of the whole team
17. Testing at Scale	the execution infrastructure and design that sustain millions of tests
18. Testing in the AI Era (this article)	the tools of generation and execution changed; the fundamental question of verification didn’t

What Can Testing Actually Guarantee — the Tools Changed, the Question Didn’t

As we saw in the first article of this series, The Prehistory of Testing, Edsger Dijkstra, following his presentation at the 1969 NATO Software Engineering Conference, wrote in his 1970 paper “Notes on Structured Programming” (EWD249):

“Program testing can be used to show the presence of bugs, but never to show their absence!”

Glenford Myers’s 1979 definition of testing as “a destructive process of trying to break a program” and Barry Boehm’s V&V distinction — “are we building the product right” (verification) versus “are we building the right product” (validation) — remain just as relevant vantage points as they were half a century ago.

LLMs have made writing tests faster, made it easier to remember the inputs we tend to overlook, and lightened the first step into untested legacy code. But LLMs haven’t erased the limit Dijkstra pointed at — if anything, through implementation-confirming tests, they’ve made the gap between “the test suite is green” and “there are no bugs” more sharply visible than ever. From SUnit to JUnit, from TDD to BDD, from mocks to property-based testing, from Selenium to Playwright, all the way to the monorepo infrastructure that sustains millions of tests — the tools and the vocabulary have been replaced, again and again, over 60 years. What’s been asked, unbroken, through all of it, is just one thing: what can testing actually guarantee?

Even as the tool shifts from human to AI, the responsibility for answering that question still rests with the human.

To the Reader

Readers who’ve made it through this series might want to go straight to the primary sources: Kent Beck’s Test-Driven Development: By Example (2002), Michael Feathers’s Working Effectively with Legacy Code (2004), Gerard Meszaros’s xUnit Test Patterns: Refactoring Test Code (2007), and Steve Freeman and Nat Pryce’s Growing Object-Oriented Software, Guided by Tests (2009). And Software Engineering at Google, which covers Google’s engineering culture, is also the primary source for the hermetic tests and test-size classification discussed in this article. In a field not yet fifty years old, these have already earned a place as classics.

The Lineage of Software Testing series ends here. From Dijkstra’s aphorism to the era of AI writing tests — if these 18 articles have conveyed how rich a history of questions lies beneath the plain act of “checking whether it works,” that will have been enough.