How the Claude Code team thinks about verification
Anthropic's Claude Code team generates 90 percent of its codebase with AI, using automated verification loops and a 39-item reliability checklist.
By Dillip Chowdary • Nov 17, 2026 • Source: claude.dev

At a glance
- Product
- Claude Code
- Author
- Addy Osmani
- Published date
- Nov 16, 2026
- Code authorship
- Roughly 90% written and reviewed by Claude
- Checklist size
- 39 reliability items triggered by diffs
- Escalation limit
- 3 failed rounds before human escalation
In this post
Anthropic engineering lead Addy Osmani published an architectural overview detailing how the Claude Code team manages autonomous software engineering at scale. As detailed in claude.dev's report, roughly 90 percent of the team's production codebase is authored and reviewed directly by Claude agents. The release outlines how engineering teams shift from manual supervision to running parallel agents by pairing language models with automated verification loops.
This technical analysis breaks down the verification framework required for enterprise coding agents, contrasting unit checks against end-to-end, property, and differential testing harnesses. The methodology serves software architects, technical leads, and developers deploying agentic coding assistants who need to enforce reliability standards without review bottlenecks. Readers will find concrete benchmarking results, testing configurations, terminal setup commands, and git branch protection mechanisms.
01Ninety percent agent authored code generation
Autonomous agent pipelines require evidence-based verification to scale beyond manual developer supervision. When human engineers inspect every code change, teams remain restricted to operating one or two agents simultaneously. Establishing automated feedback loops allows developers to delegate work across dozens or hundreds of concurrent agent sessions without sacrificing software quality.
Verification systems operate as standard software, executing tests and benchmarks repeatedly without consuming additional token budgets. When an agent experiences recurring test failures, developers codify fixes into persistent hooks or repository guidelines. This mechanism prevents regression patterns across subsequent tasks while preserving high development velocity.
02Benchmark gains from rubric grader loops
Internal evaluations conducted by Anthropic demonstrate measurable accuracy improvements when integrating automated verification into generative workflows. In structured file generation tests, deploying an independent grader model to evaluate output against a defined rubric increased task completion rates by up to 10 points over conventional prompting loops. The most pronounced performance improvements emerged on the hardest engineering problems.
| Metric | Standard Prompting | Grader Verification | Measured Impact |
|---|---|---|---|
| Structured Task Success | Baseline completion | Up to +10 points | Largest gains on hardest problems |
| Parallel Agent Scale | 1 to 2 agents | Dozens to hundreds | Significant delegation throughput |
| Verification Token Cost | High generative usage | Native software runs | Minimal additional tokens |
| Failure Resolution | Manual inspection | 3 repair rounds | Escalates to human on persistent failure |
The operational transition from prompt-based checking to software execution significantly cuts marginal inference expenses. Generated test suites run locally as native binaries, enabling continuous validation cycles across test suites without inflating operational costs. Teams maintain velocity because agents receive immediate programmatic error diagnostics rather than ambiguous warnings.
03End-to-end, property, and differential testing tiers
End-to-end testing represents the primary validation layer because full user journeys cannot easily mask underlying system failures. When agents execute scenarios like cart additions or checkout workflows, failures deliver precise visual screenshots and actionable error logs rather than generic timeouts. Teams equip agents with browser automation tools, Chrome DevTools Protocol integrations, and application harnesses guided by structured feature maps.
Advertisement
Tech Pulse Daily
Get tomorrow's pulse first
Join engineers who read Tech Pulse before stand-up. Free, weekday mornings.
Property testing validates system constraints across thousands of synthetic inputs using fast-check, Hypothesis, and proptest to prove invariants, round-trip encoding, idempotency, and equivalence. In parallel, differential testing supports migrations by running legacy and modern codebases against identical inputs to isolate unexpected divergences. Both approaches eliminate author bias during complex code rewrites.
Note: Heavier end-to-end tests must run before committing changes, as slow or flaky tests degrade agent efficiency and encourage test retries over code fixes.
04Terminal setup and the verify skill
Developers deploy the verification framework by installing the official Claude Code command-line interface across local development workstations. Teams maintain current binary installations using global package managers to ensure compatibility with recent verification protocols. The environment connects directly to local runtimes, browser extensions, and mobile device emulators.
npm install -g @anthropic-ai/claude-code
claude updateEngineers configure model selection and session parameters to ensure consistent test execution across complex verification cycles. Teams switch models dynamically inside interactive sessions or launch pinned instances directly from the terminal shell. Setting global defaults ensures that all subsequent autonomous development sessions use verified configurations.
claude --model fable
/model fable
claude config set model fable- Create verification skill Define a dedicated skill named verify to codify simulator boot and application execution scripts.
- Place skill in repository Save the configuration inside the .claude/skills/ directory as close to relevant source code as possible.
- Automate pre-commit runs Configure Claude to execute the verify skill as an obligatory smoke test before committing any changes.
- Stress test the harness Iterate and benchmark the verification harness to guarantee fast and deterministic test execution.
05Reliability checklists and branch protection rules
Failure governance rules prevent autonomous agents from looping indefinitely on failing verification tests. Claude executes automated checks and attempts autonomous remediation, but stops and escalates to human engineers if a check continues failing after three consecutive rounds. This three-attempt boundary ensures that difficult edge cases receive developer oversight without clogging unattended background processes.
Anthropic enforces a reliability checklist containing 39 discrete inspection items mapped directly to historical outages and pull requests. When evaluating a proposed diff, Claude analyzes code triggers to run between three and ten relevant checklist verifications, such as verifying whether transcript resumption handles schema migrations. Individual code owners author and refine their own domain-specific checks.
Branch architecture ensures that pull requests cannot manipulate their own evaluation rubrics during automated reviews. Evaluation tools ingest review guidelines strictly from base or default branches rather than pull request diffs, preserving validation integrity. Teams run lightweight local checks before dispatching continuous integration jobs, ensuring reviews evaluate only committed code states.
Frequently asked questions
How much code does Claude write and review on the Claude Code team?
Roughly 90 percent of the codebase on the Claude Code team is written and reviewed directly by Claude.
How does automated verification improve agent task success rates?
In internal structured file generation tests, an independent grader checking work against a rubric improved task success by up to 10 points over standard prompting.
What happens when an agent cannot resolve a failing test?
Claude runs checks and attempts to fix issues autonomously, but if a test still fails after three rounds, the system stops and escalates to a human engineer.
Why must review instructions be loaded from the base branch?
If a code change can alter the checks that evaluate it, the verification process is compromised. Review rules in REVIEW.md are loaded from the base branch so pull requests cannot edit their own evaluation criteria.
Sources
Author
Dillip Chowdary
Writes Tech Bytes coverage of AI, engineering, and the tools that actually ship. Editor of Tech Pulse Daily.
Related on Tech Bytes
Advertisement