Reviewing AI-generated code: the diff got bigger, the reviewer did not

Most of the code I approved last week, nobody typed. That is the ordinary condition of the job now, and reviewing AI-generated code turns out to be a different activity from reviewing a teammate's pull request even when the two diffs are byte-identical. Same words on screen. Different information behind them.

A human diff carries a shadow record. Every branch the author considered and rejected, every assumption they checked at 11pm before giving up and hardcoding the timeout, every place they hesitated. You cannot see any of it, but you could always feel where it was thin. Reviewers built twenty years of instinct on that residue. A generated diff has no residue. It is a first draft with the confidence of a fifth.

Generation moved the bottleneck. It did not move the reviewers.

The bottleneck moved and nobody staffed the new one

Faros AI published its 2026 AI Engineering Report on 12 April 2026, built on two years of telemetry across roughly 22,000 developers and more than 4,000 teams. The throughput side reads how you would expect. Task throughput per developer up 33.7 percent. Epics completed per developer up 66 percent. PR merge rate per developer up 16.2 percent.

Then the review side. Median time to first PR review up 156.6 percent. Average time spent in code review up 199.6 percent. Median time in review up 441.5 percent. Bugs per developer up 54 percent. Incidents-to-PR ratio up 242.7 percent. And the figure that should stop anyone who runs a team: pull requests merged without any review at all, up 31.3 percent.

LinearB's analysis of 8.1 million pull requests across roughly 4,800 teams, published 4 May 2026, arrives at the same place from a different direction. At the 75th percentile, AI-assisted pull requests ran past 400 lines against 157 for unassisted work. Agentic pull requests waited more than 16 hours on average before a reviewer picked them up, against roughly 200 minutes for unassisted work. Within 30 days, 84.5 percent of unassisted PRs merged. For AI-generated PRs the figure was 32.7 percent.

Read those together. More arrives, it arrives bigger, it sits longer and two thirds of it eventually goes nowhere. The review effort spent on the two thirds is not recovered.

Generated code is locally correct and globally naive. It has no model of what else is running.

Plausible code passes the eye test. Bad code never did.

Bad human code used to announce itself. Naming that drifted halfway through, a function that reached 300 lines because someone kept bolting on cases, a comment that contradicted the line beneath it, three levels of nesting where one would do. Ugliness correlated with danger. That correlation was doing enormous quiet work in every review process I have ever been part of.

Generated code breaks the correlation. Veracode's Spring 2026 update ran 80 coding tasks across Java, JavaScript, C# and Python against a cumulative pool of more than 150 models. Syntax correctness above 95 percent. Security pass rate 55 percent. Java came in at 29 percent. Cross-site scripting, CWE-80, passed 15 percent of the time. Log injection, CWE-117, passed 13 percent. So: almost always well-formed, wrong on security roughly half the time. Surface and substance have come apart.

GitClear's January 2026 study of 623 million changes from 2023 to 2026 measures the same split structurally. Block duplication rose from 40.3 per million changed lines in 2023 to 73.0 year to date in 2026, an 81 percent increase and the highest on record. Within-commit copy/paste went from 9.4 percent in 2022 to 15.7 percent in the first half of 2026. Moved code, the signature of real refactoring, collapsed from 21 percent in 2022 to 3.8 percent. Error-masking constructs up 47 percent.

Sit with that last one. A generated try/except that swallows the exception and returns a default reads, on a fast pass, as defensive programming. It is a muted alarm. It will look responsible in review and it will delete your only signal in production.

Stack Overflow's 2025 developer survey put the top AI frustration at 66 percent of respondents: solutions that are "almost right, but not quite." Second, at 45.2 percent, that debugging generated code takes longer. Only 3.1 percent said they highly trust the output. The people using these tools every day already know. They just have no process that reflects it.

What to actually check when reviewing AI-generated code

The old heuristics were proxies for author attention. Those proxies are gone, so what is left is the tedious explicit list. You have to run it deliberately, because nothing in the diff will prompt you the way a sloppy human commit would.

Boundary conditions. Empty collection, single element, off-by-one at the top of the range, the maximum the type can hold, the timezone at the date boundary. Public code demonstrates the happy path, so that is what got learned.

Error paths, and not whether errors are handled. Whether they are handled correctly. Read every catch and ask what state the system is in after it runs. A retry with no cap. A fallback returning a cached value with no staleness check. An error swallowed at a layer that had no business deciding.

Resource cleanup on the failure branch. The success path closes the file. Does the path that throws on line 40. Connection pools, file handles, mutexes, temp directories and database cursors. This is where I find the most real defects, and it is invisible to the eye test precisely because the cleanup code exists, just not on every exit.

Concurrency. Shared mutable state, a lock held across an await point, check-then-act on something another thread can change underneath you. Generated code is locally correct and globally naive. It has no model of what else is running.

The silent omission, which is the hardest of all. Ask what the ticket required that the diff does not contain. Generated code fails by quietly narrowing the problem to the case you described in the prompt, dropping the two you assumed were obvious. There is no TODO. There is no ragged indentation. The absence looks exactly like completeness.

Interface provenance. Method signatures that were never defined, config keys nothing reads, a library whose API changed two minor versions ago. Grep it. Do not trust it because it compiles in your head.

Review fatigue is a physical limit, not an attitude problem

SmartBear spent ten months analysing 2,500 reviews covering 3.2 million lines of code at Cisco Systems. It remains the largest published dataset on human code review, and the numbers have not aged. Defect detection falls off sharply above 400 to 500 lines per hour. A review of 200 to 400 lines over 60 to 90 minutes yields 70 to 90 percent defect discovery. Detection rates collapse after roughly 90 minutes of continuous reading.

Now put a 408-line generated pull request in front of that reviewer, and then another, at a merge rate 16.2 percent higher per developer. Honest daily capacity is two careful reviews, maybe three. The queue does not care about the study.

What degrades first is not accuracy, it is depth. The reviewer keeps approving. The approvals stop carrying information. "Looks fine" is what a saturated reviewer says, and it has a diagnostic signature you can watch for: comments about naming and import order on a diff that changes locking behaviour. When your team's review comments drift toward style, you are already past capacity and the process has become decorative.

Let the machines review first

Human attention is now the scarce input in the pipeline. Spend none of it on anything a tool can decide.

Static analysis blocking on every diff, not advisory. Semgrep and Snyk in the required checks. The Veracode CWE distribution is the argument: the failures cluster in a small, well-known set of patterns, injection, unsafe crypto defaults and unescaped output, which is exactly the class of thing pattern matching is good at and exactly the class of thing a tired human skims past.

Tests as first reviewer, with one norm change that matters more than the rest. The test has to be shown failing against the old code before it counts. Generated tests written after generated code test what the code does, not what it should do. I have watched a suite go green against an implementation with an inverted condition, because the same context produced both sides.

Mutation testing on the modules where being wrong is expensive. Coverage tells you the line executed. Mutation tells you whether any assertion would have noticed if the line were wrong. Against generated code that distinction stops being academic.

Property-based tests wherever the domain has real invariants. The boundary cases you would otherwise enumerate by hand are what a generator produces for free, and boundaries are the documented weak spot.

The strongest objection, and what it misses

The serious counter-argument is that reviewers were never good at this. Approvals were rubber stamps long before models existed. Everyone has watched a 900-line refactor merge in four minutes with a thumbs-up emoji. On this reading, blaming generated code for broken review romanticises a process that was already theatre.

I will go further and undercut my own side. The METR randomised trial published on 10 July 2025 is the most-cited evidence that AI slows experienced developers: 16 developers, 246 tasks, an average of five years on the repositories in question, forecasting 24 percent faster, self-reporting 20 percent faster and measuring 19 percent slower. On 24 February 2026 METR published that it was changing the experiment design, citing selection bias serious enough to undermine the follow-up: developers refusing to participate at all if they could not use AI, 30 to 50 percent of them declining to submit tasks they thought AI would accelerate and recruitment worsening after pay dropped from $150 an hour to $50. Their late-2025 results put the original developers at 18 percent slower with a confidence interval running from 38 percent slower to 9 percent faster, and newly recruited developers at 4 percent slower with an interval from 15 percent slower to 9 percent faster. The headline number is much softer than the internet made it.

Both concessions are true. Neither one rescues the objection. Review was theatre when authorship was the constraint, and the theatre was affordable because a human had already done the thinking somewhere upstream of the diff. The rubber stamp was ratifying a judgement that had been made by someone. Now no one made it, so the stamp is the entire process rather than the last step of it. Same ritual, load-bearing for the first time.

And METR is not the evidence for the review claim anyway. METR measured how fast individuals finish tasks, which is contested. Faros and LinearB measured queue times, pickup delays, PR sizes and merge rates from telemetry, which is counting, not asking people how fast they felt.

Norms that have to change, concretely

Cap diff size in CI. Anything above 400 lines splits, or ships with a named exemption and a reason. This is the highest-return single change available, because it is the only one that puts the SmartBear ceiling into the pipeline instead of into a wiki page nobody reads.

Make the author write the review guide. A generated PR arrives with a description stating what was verified by running it, what was only read and what was not checked. Provenance as a required field, not a confession volunteered by the conscientious.

Give reviewers a third verb. Approve and request-changes are the only exits, so saturation has nowhere to go except into approvals. "I read the migration and not the tests" needs to be a legitimate, recordable outcome.

Staff review as work. If task throughput per developer is up a third, review hours have to appear in the plan rather than being absorbed by whoever is most agreeable.

Stop rotating the dangerous modules. The payment path, the schema migrations, the auth boundary: same two reviewers for six months. The context that the generated diff no longer carries has to live in a human somewhere, and rotation is how you guarantee it lives nowhere.

The datasheet always lies

Eight years in Shenzhen has left me with one comparison I cannot shake. Buy a reel of capacitors in Huaqiangbei and nobody, at any point in the chain, treats the printed spec as the answer. Every factory floor I have walked through runs incoming quality control on components that arrived with a passing test report stapled to the box. Not because the supplier is assumed dishonest. Because the report was produced by a process with its own interests, so you sample anyway. The report is a claim. The part is a fact.

Software went the other way. We built review on the premise that the author suffered. Someone stared at the problem, chose badly, backed out and chose again, and the diff in front of you is the sediment of that. Approving it was partly a judgement about the code and largely a judgement about the person who had already judged it. That premise died somewhere around 2024. The ritual outlived it by two years and counting.

Here is the part I genuinely cannot resolve. Every counterweight we reach for, generated tests, generated review comments, generated static analysis rules, comes from roughly the same distribution as the code it is checking. When Alibaba released Qwen3-Coder-480B-A35B on 23 July 2025 under Apache 2.0, I read the Chinese release notes the day they went up, and the thing that struck me was not the benchmark table. It was that the same weights would be writing both the implementation and the test that certifies it.

Incoming quality control works because the inspector is not the supplier. We have built a supplier who also writes the certificate, handed it to a review process designed to read handwriting, and called the throughput number a win.

Tools referenced

Semgrep, reviewed here: Semgrep review.

Snyk, reviewed here: Snyk review.

Cursor, reviewed here: Cursor review.

GitHub Copilot, reviewed here: GitHub Copilot review.

Devin, reviewed here: Devin review.

Claude, reviewed here: Claude review.

Sources

METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (10 July 2025): https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/

METR, We are Changing our Developer Productivity Experiment Design (24 February 2026): https://metr.org/blog/2026-02-24-uplift-update/

Faros AI, The AI Engineering Report 2026: Ten Takeaways (12 April 2026): https://www.faros.ai/blog/ai-acceleration-whiplash-takeaways

LinearB, 8 million pull requests reveal where engineering productivity breaks down (4 May 2026): https://linearb.io/blog/8-million-prs-engineering-productivity

GitClear, The Maintainability Gap: 2026 AI Code Quality Research (January 2026): https://www.gitclear.com/the_ai_code_quality_maintainability_gap

Veracode, Spring 2026 GenAI Code Security Update: https://www.veracode.com/blog/spring-2026-genai-code-security/

Stack Overflow 2025 Developer Survey, AI section: https://survey.stackoverflow.co/2025/ai

SmartBear, Code Review at Cisco Systems case study: https://static0.smartbear.co/support/media/resources/cc/book/code-review-cisco-case-study.pdf