Claude Code team explains how it checks AI-written code—and when humans step in
A new engineering post describes checks that code changes cannot rewrite, escalation after repeated failures, and evaluations built around past bugs.
Loading page…
A new engineering post describes checks that code changes cannot rewrite, escalation after repeated failures, and evaluations built around past bugs.
Listen to this story
In a Nov. 16, 2026 engineering post, Addy Osmani describes Claude Code’s verification system, where the team says roughly 90% of its code is written and reviewed by Claude. A separate rubric-based grader lifted success on structured-file generation by up to 10 percentage points, but that result is limited to that task. The workflow centers on running real user journeys, keeping review rules on the base branch, and escalating checks that remain failing after three repair rounds—an operational model for delegating code without treating AI review as self-validating.
For migrations, differential tests compare old and replacement systems on random inputs or sampled production traffic, while allowing explicitly intended differences.
The review team tests bug detection on reconstructed historical pull requests and uses stable, merged changes as controls against overflagging.
New rules drawn from human comments or reversals must recur in at least two pull requests, survive attempts by three agents to disprove them, and receive human approval.
Delegating more coding does not eliminate the work of checking it. In a November 16, 2026 engineering post, the Claude Code team describes how it handles that burden—and reports an internal result: a separate grader improved structured-file-generation success by up to 10 points. That finding concerns a specific task, not a measured improvement across all software development.
The post, by Addy Osmani on claude.dev, says roughly 90% of the team's code is written and reviewed by Claude. Its approach is not simply to add more AI reviewers. It combines running the software, protecting review rules from the changes they judge, and handing persistent failures to people.
The internal experiment used a separate grader to check Claude's work against a rubric, or scoring guide. The largest gains came on the hardest problems, compared with a standard prompting loop. The post presents that result as evidence for giving agents a feedback loop, rather than relying on instructions alone.
For coding, Osmani prioritizes end-to-end checks: running an actual user journey through the application. A small, isolated test can pass while the feature still fails. The post recommends giving Claude a reusable verify skill containing instructions for starting and exercising the app, then making that routine fast and reliable.
Two other test types address gaps that a developer—or the agent writing the code—might miss:
Human attention enters at a defined point. Claude runs checks and attempts repairs, but if a check still fails after three rounds, it stops and brings in a person. Code owners write their own checks and refine them to flag the issues they would flag themselves.
Review instructions live in REVIEW.md on the base or default branch—the established code, not the proposed changes. That prevents a submission from rewriting its own judging criteria. Reviews also apply to specific commits, the saved versions of a change, rather than code that keeps moving during inspection.
A passing review on an out-of-date commit means nothing.
Addy Osmani, writing on claude.dev
The team favors tuning one reviewer over stacking another, more expensive reviewer on top. Successive passes receive context from earlier findings. A separate reliability checklist contains 39 items tied to recognizable code changes; Claude typically checks only the three to 10 whose triggers match, rather than applying every lesson to every submission.
The post also describes an evaluation built from past incidents and reverted changes. The Code Review team reconstructs the code as it stood at review time and scores findings only when they correctly identify the known bug. Stable, merged changes serve as a control against flagging problems everywhere. Tuning examples remain separate from the fixed test set.
They measure bug detection at three stages: initial findings, after verification, and final review. That separates failure to discover a bug from losing a valid finding later. Proposed new rules drawn from human comments and reversals must recur in at least two pull requests; three agents try to disprove them before human approval.
Even enforcement has an escape hatch. A commit gate can log missing verification or block it, but after two blocked commits in a session, the next is allowed through so Claude does not get stuck. The team also records how each commit was verified—including “none”—so unchecked work remains visible.
Loading discussion...
Join the conversation
Explain when an escape hatch is sensible—or defeats the safeguard.
Be the first to share a perspective or an experience.
Reader comments
Newest comments first. Replies stay oldest first.