Your AI code review is a rubber stamp

Review debt is the gap between the code your team produces and the code anyone actually examined. AI review tools have made that gap comfortable to ignore, because a pull request with an automated comment sitting on it looks reviewed. When a tool comments and no human ever responds, the review happened in form and not in substance. Below: what the measured numbers actually say, why this is a structural failure rather than a discipline problem, and the one change that fixes it.

Last reviewed: August 2026

Table of contents

• What is review debt?
• Why does an AI code review become a rubber stamp?
• Does AI-generated code actually need more review?
• What does "reviewed" actually mean?
• What a verifiable acceptance criterion looks like
• Who decides?
• What I changed in my own build lane this week
• Where this fits, and what to do next
• References

What is review debt?

Tech debt is visible. Somebody wrote the shortcut, somebody remembers it, and it shows up as friction the next time you touch that file.

Review debt does not behave that way. It accumulates in code that nobody objected to, because nobody read it. There is no artifact. The pull request is green, the automated reviewer left three comments about naming, someone clicked approve, and the change is now indistinguishable from code that a senior engineer walked through line by line.

The measured version of this is uncomfortable. Cortex's 2026 benchmark found pull requests per author up 20% year over year while incidents per pull request rose 23.5% and change failure rates climbed roughly 30%. More changes, each one likelier to break something. That is what review debt looks like on a dashboard, well after the reviewing decision was made.

Why does an AI code review become a rubber stamp?

The obvious explanation is that people got lazy. The obvious explanation is wrong, and the real one is worse.

Qodo's product telemetry, published alongside their 2025 survey of 609 developers, reports that when an AI review tool is enabled, 80% of pull requests receive no human comment or review at all. Not fewer comments. None.

Read that as a design outcome rather than a moral failure. The tool was installed to catch things. It does catch things. It comments on real problems, and the comments are usually correct. Every one of those correct comments teaches the team that the tool is handling it, and the marginal value of a human opening the diff appears to drop. The team is responding rationally to evidence.

The same survey found that developers who rarely encounter hallucinations are 2.5 times likelier to feel very confident shipping AI-generated code, 24% against 9%. So the trust is earned. The tool genuinely did get better, and confidence rose to match. That is precisely the problem: the mechanism that produces the rubber stamp is the tool working well, not the tool working badly. Nothing about a better model reverses it. Only 3.8% of the developers Qodo surveyed reported both low hallucination rates and high confidence shipping without human review, which tells you how few teams have actually landed in the place everyone assumes they are heading.

Worth stating plainly: Qodo sells AI code review, and that 80% is telemetry from their own product with no published methodology. It is the closest thing to a direct measurement of this behaviour that exists publicly, and it comes from a company with an interest in the answer. Both things are true.

Does AI-generated code actually need more review?

Yes, and the pattern in the data is more specific than "AI writes worse code."

Veracode tested over 100 large language models across Java, Python, C# and JavaScript for their 2025 report. 45% of generated code samples introduced an OWASP Top 10 vulnerability. Java was worst at a 72% security failure rate. Against cross-site scripting, the models failed to defend in 86% of relevant samples.

The finding that should change how you think, though, is this one, in their words: while the models got better at writing functional or syntactically correct code, they were no better at writing secure code, and security performance stayed flat regardless of model size or training sophistication.

Two capabilities that everyone assumed were the same capability turn out to be separate. Code that runs and code that is safe improved on entirely different curves, and only one of them improved. Waiting for a better model does not close a gap that better models have not been closing.

CodeRabbit's December 2025 analysis of 470 open-source pull requests points the same direction from a different angle: AI-co-authored pull requests carried about 1.7 times more issues overall, 10.83 per pull request against 6.45 for human-only ones, with security issues up to 2.74 times higher and improper password handling the most prominent pattern. Their sample is unbalanced and their authorship labels are inferred rather than confirmed, which they say openly in the report, and they also sell AI code review. Treat it as directional.

Then there is the slower failure. GitClear examined 211 million changed lines from 2020 to 2024 and found that lines associated with refactoring fell from 25% of changed lines in 2021 to under 10% in 2024, while copy-pasted lines rose from 8.3% to 12.3%. For the first time on record, copy-paste overtook moved code. Nothing there is a bug. It is a codebase quietly becoming harder to change, one reasonable-looking pull request at a time, and no review gate is even watching for it.

The two least commercially interested voices in this space reached the same conclusion independently. DORA's 2025 report describes AI's primary role as an amplifier, magnifying an organization's existing strengths and weaknesses. Cortex, from an entirely different dataset, calls it an indiscriminate amplifier that takes your existing engineering practices, good and bad, and magnifies their impact.

If AI amplifies whatever process you already had, and your process was "a human eyeballs the diff and forms an impression," then you now have that, amplified. An impression, formed faster, about more code.

What does "reviewed" actually mean?

Here is the question almost nobody on a team can answer: what specifically has to be true for this change to count as reviewed?

Not "someone looked." What was checked, against what, and how would we know if the check had failed?

Most teams cannot answer, which means their review gate has nothing concrete to compare against. And a gate with no standard behind it degrades to whether the reviewer felt uneasy. That works when the reviewer wrote most of the code themselves and holds the system in their head. It stops working the moment volume climbs, and volume has climbed.

The clearest statement of the fix I have seen this year is on the front page of a marketplace, not in an engineering blog. Bounty is an open marketplace where AI agents compete for posted work and earn. Its pitch to the person posting the job is one sentence:

If you can say what done looks like, you can post it.

That is the whole thing. Their model only functions because the buyer states acceptance criteria up front, and an automated oracle verifies the submitted work against those criteria before escrow releases. Pass and the payment goes through. Fail and the money comes back. The verification is not a courtesy. It is load-bearing, because without it nobody would fund a stranger's agent to do anything.

Most engineering teams have never had to be that explicit, because a human reviewer could always fall back on judgment. Once the volume of change outruns the amount of judgment available, that fallback quietly stops being there, and nothing replaces it.

What a verifiable acceptance criterion looks like

Acceptance criteria describe success. That is the easy half, and most teams already write them.

The half that catches real defects is a line naming the specific way a plausible, passing implementation could still be wrong. In my own specs it is a required field, sitting directly beneath the acceptance criteria, and a story is rejected before any work starts if it is missing. The shape:

acceptance:
  - a failed assurance check surfaces in the response payload
  - the response includes the check id and the reason
wrong-if:
  - a check that errored during execution is reported to the caller as passing

The acceptance criteria are satisfiable by an implementation that treats every non-pass as a pass. It would look correct in a demo. The tests would be green. The wrong-if line is what stops it, because it names the misunderstanding that a competent implementer is most likely to make.

That single line does something a checklist cannot. It converts review from "does this look right" into "here is the specific scenario I am going to try to trigger." A reviewer with that line has an assignment. A reviewer without it has an impression.

Underneath sits a rule I would now put in any system that checks anything. Every check reports one of three values: pass, fail, or not-run with a reason. Never two. The moment "did not run" can quietly render as "fine," your green dashboard stops meaning anything, because you cannot distinguish a check that passed from a check that never executed.

In the clinical verification engine I build, that rule lives in the database rather than in a convention, as a constraint refusing to store a not-run row without a reason. A rule enforced by a schema survives a bad afternoon. A rule enforced by a paragraph in a document does not.

Unknown and safe are different words.

Who decides?

Three systems I have looked at closely this year, built by different people for unrelated purposes, all landed on the same rule.

Bounty's oracle verifies the work before money moves. The agent that did the work does not get to grade it.

The clinical verification engine treats the language model as something that can draft, argue and explain, but never decide. Deterministic code owns every verdict and every number. A model can make a case for a score. It cannot set one.

And in my own build lane, the agent that writes code never issues the verdict on it. The verdict comes from actual test-runner output, a security checklist, and a deliberate attempt to break the thing, never from any model's opinion that the code looks fine.

Convergence like that usually means the constraint is structural. A judge that shares an author, a model, or an incentive with the thing being judged is not a judge. It is the same opinion, expressed twice, with more confidence the second time.

Which is what an AI review tool becomes when nobody reads its output. The generating model and the reviewing model are drawn from the same training distribution and are prone to the same blind spots. Left unread, that is not a second opinion. It is the first opinion wearing a badge.

The open question I keep circling is what happens if you take a marketplace like Bounty and remove the money. Strip out the payment and the structure that remains is an evaluation environment: a task, an environment, tools, constraints, ground truth, an agent, and a verifier. Run the same task against several agents with the same verifier and you have a controlled comparison rather than a vendor benchmark. Run it a hundred times and you learn what an agent does on its bad days, which is the only number that matters before you let it near anything real. Score the execution trace and not just the output, and you can also see whether it reached the right answer for a reason that will hold next week. I have not built this. It is where I think the interesting work is.

What I changed in my own build lane this week

I spent the past week rebuilding the part of my agent fleet that writes and checks code. Roles, not names: an implementer, a reviewer, a QA agent, a security analyst, and an engineering lead who owns the gate.

The gate has four conditions, and all four have to hold:

1. The reviewer returns a pass.
2. The QA agent returns a pass with real test-runner counts quoted. Not a summary. The numbers the runner printed.
3. Security returns a pass.
4. The branch is a work branch, never a default branch.

The parts I would actually defend are the ones that make the gate harder to satisfy.

The reviewer's last pass is falsification. It reads the wrong-if line and tries to trigger it on purpose, constructing the input that would make a passing implementation wrong. If it cannot construct that scenario, it is not allowed to record a pass. It has to write "wrong-if unverifiable because X," which is a worse-looking outcome and an honest one.

If the change touches an area that has a specialist reviewer and that reviewer did not run, the verdict is `INCOMPLETE`. Never a clean pass with a missing piece.

Rework is capped at two cycles. A third failure stops the machine and hands all three reports to me, because at that point the story is wrong, not the code.

And nobody in the fleet can push. Not the implementer, not the lead, not me acting through them. The gate runs, it reports "cleared, awaiting authority," and a human moves the branch. I wrote a proposal to grant push rights and then deferred my own proposal, because a push grant scoped by an agent's definition is enforced by a paragraph, and a push grant scoped by branch protection is enforced by a server that does not have a bad day. Until the server-side guard exists, the only thing between a misrouted push and production is an agent behaving well, and that is not a control.

The most useful thing that happened all week was my own audit script embarrassing me. It reported zero stale documents while a specification sat five days out of date. The staleness checker iterated a hardcoded list of two filenames, and the stale file was not one of them. The check was passing because it was not looking.

A check with a blind spot reads as a clean result, which is worse than having no check at all. No check leaves you appropriately nervous. A false pass sends you home.

That is the same failure as the rubber stamp, one level up. I had built a verification layer and then trusted its green light without asking what it actually examined. The fix in both cases is identical, and it is not "try harder." It is to make the check state what it covered, and make it say so out loud when it could not run.

Where this fits, and what to do next

Five questions, worth asking before the next AI-assisted change ships:

1. What would make this wrong? Write it down in the ticket, before implementation, in one sentence. If nobody can answer, the work is not specified yet.
2. Can a machine tell whether that happened? If it needs a human to feel uneasy, it is not a check.
3. What does the tool actually cover? An AI reviewer that reads the diff cannot see architectural drift, duplication trends, or the thing you forgot to write.
4. What happens when a check cannot run? If the answer is "it goes green," fix that before anything else on this list.
5. Who decides? If the thing that wrote it also cleared it, nothing has been reviewed.

None of this is about using less AI. I run a fleet of agents that write code daily, and I am not slowing that down. The volume is fine. The volume was never the problem. The problem is that most teams scaled generation and left verification exactly where it was, then read the green check as though it still meant what it meant in 2022.

If you are working through what this looks like for a team rather than a solo build, that is the sort of thing charmthirteen's workforce training covers.

References

1. Qodo. *State of AI Code Quality*, 2025. Survey of 609 developers. https://www.qodo.ai/reports/state-of-ai-code-quality/. Source of the 80% no-human-review figure, which is Qodo product telemetry rather than a survey result, and of the 2.5x confidence gap. Qodo sells AI code review. *My take: the most directly relevant number in this space, and the one I would most like to see replicated by someone with nothing to sell.*
2. Veracode. *2025 GenAI Code Security Report*, 30 July 2025. Over 100 LLMs across four languages. https://www.veracode.com/blog/genai-code-security-report/. 45% of samples introduced an OWASP Top 10 vulnerability; security pass rates flat while syntax improved. *My take: the strongest finding here, because it separates two capabilities everyone treats as one.*
3. Cortex. *Engineering in the Age of AI: 2026 Benchmark Report*, 12 November 2025. Survey of 50+ engineering leaders plus development metrics. https://www.cortex.io/post/ai-is-making-engineering-faster-but-not-better-state-of-ai-benchmark-2026. Incidents per pull request up 23.5% against a 20% rise in pull requests per author. *My take: small sample, so treat the exact percentages loosely and the direction seriously.*
4. DORA. *State of AI-assisted Software Development*, 2025. Presented by Google Cloud. https://dora.dev/research/2025/dora-report/. AI as an amplifier of an organization's existing strengths and weaknesses. *My take: the least commercially interested source on this list, and it agrees with the most interested ones.*
5. GitClear. *AI Copilot Code Quality: 2025 Look Back*. 211 million changed lines, 2020 to 2024. https://www.gitclear.com/ai_assistant_code_quality_2025_research. Refactoring-associated lines fell from 25% to under 10%; copy-pasted lines rose from 8.3% to 12.3%. *My take: the only source measuring the slow failure rather than the immediate one.*
6. CodeRabbit. *State of AI vs Human Code Generation Report*, 17 December 2025. 470 open-source pull requests, 320 AI-co-authored and 150 human-only. https://www.coderabbit.ai/blog/state-of-ai-vs-human-code-generation-report. Roughly 1.7x more issues overall. Their own stated limitation: "we cannot guarantee all the PRs we labelled as human authored were actually authored only by humans." *My take: directional. Authorship is inferred, the sample is unbalanced, and two widely circulated figures from this report are not printed anywhere in the public version, so I have not used them.*
7. Bounty. https://trybounty.ai/. An open marketplace where AI agents compete for posted work, with an automated oracle verifying results against stated acceptance criteria before escrow releases. *My take: the clearest working statement of "define done, then verify it" I have seen, and the orchestration pattern is more interesting than the payments.*

Related in this series

This is the first article in a cluster on shipping AI-generated code responsibly. Further pieces on review debt, writing acceptance criteria for AI-assisted work, and choosing between AI coding tools are in progress and will be linked here as they publish.

In the meantime: I deleted my entire AI team while watching Summer House covers the governance vault that this build lane runs on, and why the fleet's memory lives off-machine.