tokenise by Rosvetic
Development Analysis

One pull request, fifty review rounds: what Anthropic changed in its AI reviewer

The Claude Code team's account of its own review pipeline is mostly about one failure, an agent and a reviewer that never stop, and the cheap fixes that cut it.

6 min read

Hundreds of fine teal and off-white lines sweeping in from both sides and tightening into one bright twisting knot at the centre

Anthropic’s Claude Code team has published an account of how its pull requests get reviewed, and the most useful part isn’t the pipeline diagram. It’s one number: a single CLI pull request went through more than fifty review rounds overnight before its author stopped it.

The post, “How the Claude Code team does code review” by Addy Osmani, Morgan Westlee Lunt and Qing Wang, went up on 15 October 2026. If you run an AI reviewer against AI-written code, the fifty-round story is the one to read twice, because it’s the failure you’ll hit first.

The context

The authors say Claude writes about 90% of Claude Code’s own code. Between June and September, merged pull requests more than doubled, and the share that got reverted fell by more than 40%. In Anthropic’s monorepo, daily landed changes went from about 2,500 in May to more than 30,000 in September.

Their stated goal: most pull requests in the Claude Code repos merge with at most one human in the loop, with no rise in reverts or incidents. Human time is the scarce resource, so they track human approvals per pull request, human comments, and how long a PR waits on a person. Revert and incident rates are the guardrail. In their words, if those rise, human time has been moved, not saved.

The layers, briefly

The pipeline has seven layers, and most are familiar. Before the PR, Claude runs tests, the build and the linter, and a second agent reviews the diff in a fresh context. On the PR, required CI runs, then an AI review (parallel bug-hunting agents, with a separate group voting on which findings survive), then a security review that can block merge. Human approval scales with risk. A merge queue, feature flags and staged releases sit behind that.

Three details are worth lifting out.

Risk tiers come from the change, not the author. Docs and additive changes behind a flag need no human once checks are green. Ordinary product changes need one person. The highest-risk code needs two. A classifier sets the tier, a second classifier asks whether a person is needed anyway, and some paths, including any CI workflow file, always get one.

An agent’s approval never counts as a person’s. The person directing an agent-written PR counts as a required reviewer, and an approval covers one specific commit.

Findings are tagged Important, Nit or Pre-existing, and every finding must be fixed or resolved with a reason. That last rule is what makes the next problem possible.

The loop that wouldn’t end

Put an AI reviewer in front of an AI author and require every finding to be addressed, and you’ve built a feedback loop. The author fixes, pushes, the reviewer reads again and finds something else. The post says one CLI PR went round more than fifty times overnight.

The diagnosis is the interesting bit. Running git blame over the flagged code, the team found that about 59% of it had been there since the first pass. Later rounds weren’t catching bugs the fixes introduced. They were surfacing old issues the reviewer hadn’t mentioned the first time, one at a time.

That’s a property of how these reviewers work, not a quirk of one PR. A reviewer that reports a handful of findings per pass will report different ones on the next pass over the same code. Each report costs the author a commit, a CI run and another review. Noise is expensive when the author is an agent, because an agent will faithfully act on every nit.

The fixes

The team lists five changes:

  • A round cap. After three rounds the reviewer stops listing findings and asks the author to reconsider the approach.
  • Diff-only re-review after the first few passes, so the reviewer looks at what changed instead of re-reading the whole PR.
  • Flagging the bug class, so every instance of a mistake is fixed in one round rather than one line at a time.
  • No repeats of findings already posted.
  • Nightly review runs over existing code, so old issues surface on a schedule instead of leaking into unrelated PRs.

The reported results on internal PRs, before and after: passing after one review-and-fix round went from 65% to 78%, PRs needing five or more rounds fell from 8.9% to 1.5%, and the mean number of reviews to pass dropped from 2.08 to 1.32.

These are Anthropic’s own figures on its own repos, with a reviewer it also built. Treat them as a direction, not a benchmark. The mechanism is believable, though, and none of the fixes needs Anthropic’s tooling.

Incidents as the reviewer’s test suite

The other idea worth copying is how they tune the reviewer. They collected past pull requests that caused incidents or reverts, along with the fix-forwards, and had engineers confirm each bug and where it lived. The reviewer runs against that set daily, and every change to it is scored against all the cases.

They also sort review comments by what happened to them: fixed, disputed, marked intentional, or didn’t matter. And they run the reviewer over known-clean PRs to count false alarms. The stated priority is catching real bugs first and cutting review rounds second.

That’s a regression suite for a reviewer. Without one, a prompt tweak that feels cleaner is a guess.

What to do with this

You don’t need 30,000 merges a day to hit the convergence problem. A team with one agent author and one AI reviewer can see it in a week. Two things in the post are directly usable, plus one measurement.

First, cap the rounds. If your reviewer is Claude Code Review, the knobs are the review level (Quick, Standard or Deep, set per repo by an admin) and a REVIEW.md that lowers severities and caps nits. The earlier Tokenise piece on REVIEW.md covers that file and a CI gate. The round cap itself isn’t something that post configures, so for now it’s a convention: tell the author agent, in CLAUDE.md, to stop after three rounds and report what’s still open instead of looping.

## Review rounds
If a pull request has had three review rounds, stop fixing findings one by one.
Summarise what the reviewer keeps raising, say whether the approach is the
problem, and ask before pushing again.

Second, route by path, not by author. Standard GitHub features cover most of the tiering. A CODEOWNERS file with the paths that always need a person, plus branch protection requiring code-owner review and dismissing stale approvals on new commits, gives you the “an approval covers one commit” rule and the “CI workflow files always need a human” rule.

# .github/CODEOWNERS
/.github/workflows/   @your-org/platform
/src/Billing/         @your-org/payments @your-org/security
/src/Auth/            @your-org/security

Third, count rounds. Pull the last month of merged agent-authored PRs and count review rounds per PR. If the median is two and the tail is five or more, you have the problem the post describes, and you know the baseline before you change anything. If you keep a list of the incidents your reviewer should have caught, replay it whenever you edit its instructions. Ten cases is a start.

One caveat on the whole approach. The post is candid that Anthropic has no circuit breakers yet that tighten approval when revert or incident numbers move. Lighter paths for low-risk changes only stay safe if somebody is watching the revert rate, and that part is manual.

This week: add the three-round rule to your CLAUDE.md, and count rounds on your last 20 merged PRs.