METR's 1.9x AI R&D speedup at Anthropic, and why it ranges from 1.1x

Independent AI evaluation group METR completed an 11-week assessment of Anthropic's internal research pace between August 12 and October 29, 2026, finding that the lab's capability growth accelerated to roughly 1.9 times its prior trajectory since October 2025. According to theainewsreport.com's report, the study relied on internal data provided by Anthropic and concluded that capability gains expanded from approximately 15 index points annually before October 7, 2025, to about 29 points per year afterward. The evaluation team estimated subjectively that around 80 percent of the observed October 2026 uplift stems from AI tools assisting research, while staff internal surveys registered even higher perceived individual velocity.

This analysis breaks down the mathematical modeling behind the reported 1.9x figure, why standard statistical choices spread the true acceleration between 1.1x and 2.9x, and what constraints limited external verification. It is intended for machine learning engineers, research directors, and policy analysts tracking frontier lab velocity, evaluation methodology, and the empirical impact of AI self-assistance. While Anthropic provided data for the assessment, the company exercised unilateral redaction rights over specific benchmark names and canceled a planned compute-allocation audit before publication.

METR's 1.9x AI R&D speedup at Anthropic: what actually changed

The assessment examined progress across unreleased frontier architectures labeled Model 2, which finished training in July 2026, Model 3, developed from August to September, and Model 4, completed between September and October 2026. These internal checkpoints were evaluated alongside public models including Mythos Preview and Haiku 5.5 to measure historical trajectory. Rather than testing live query interfaces for unreleased models, METR worked from Anthropic's logged benchmark results, 15 staff interviews, engineering headcount figures, pull request volumes, training compute allocations, inference expenses, and internal efficiency studies. The full findings were compiled into an unpublished 100-page internal document.

The central calculation used Anthropic's AECI metric, an implementation of the Epoch Capabilities Index built on Epoch's public code repository. The framework scores models similarly to standardized tests by assigning each model an unobserved latent ability score while assigning each test task a difficulty parameter and a discriminability factor. Fitting linear trajectories before and after the fixed October 7, 2025 breakpoint produced the 15-point and 29-point annual slopes, yielding the headline 1.9x ratio with a 95 percent confidence interval spanning 1.5x to 2.8x. A slope-change model produced an error of 1.9 points versus 3.9 for a one-time step change, winning 81 percent of bootstrap iterations.

METR's 1.9x AI R&D speedup at Anthropic: how it works

The resulting acceleration multiple depends entirely on the mathematical link function selected to translate latent ability scores into benchmark performance percentages. A link function defines the distribution of problem difficulties within an evaluation, determining whether most problems sit near the floor or near the ceiling. METR's baseline calculation employed a standard symmetric logistic curve, which produced the 1.9x outcome. Alternative standard link functions produced widely divergent rates on the exact same dataset, ranging from 1.1x for a complementary log-log curve to 2.9x for a log-log curve, alongside 1.8x for probit, 2.6x for learned Stukel, and 2.8x for cauchit.

Because complementary log-log functions assume many trivial tasks, late-stage score improvements require less incremental ability, dropping the calculated acceleration to 1.1x. Conversely, log-log functions assume benchmarks contain predominantly hard tasks, meaning late-stage gains require massive ability increases and push the acceleration to 2.9x. Across the dataset, these curves fit the out-of-sample data with nearly identical predictive accuracy, showing R-squared values hovering between 86 and 87 percent. This statistical parity prevents the raw numbers from isolating an unambiguous winner between 1.1x and 2.9x.

METR's 1.9x AI R&D speedup at Anthropic: why it matters

The study highlights a sharp divergence between individual engineer impressions and institutional research output. Internal Anthropic Slack polls tracking how much models speed up daily work rose from 1.2x for Claude 3.7 Sonnet in January 2025 to 12x for Model 4 in October 2026. METR classified these emoji-based staff votes as very loose evidence because respondent samples were self-selected and individual coding efficiency does not bypass physical research dependencies. Overall lab progress remains bound by compute constraints, physical experiment runtimes, and architectural decision cycles that do not accelerate linearly with individual coding speed.

Interviews with staff members reflected a more moderate view that aligned closely with measured outcomes. Coded by METR with analytical assistance from Claude, the 15 interviews yielded two affirmative responses, three leaning affirmative, four borderline ratings, zero leaning negative, and two negative assessments. Regarding the origins of the speedup, METR's subjective consensus attributed roughly 80 percent of the October 2026 acceleration to AI systems, supported by steady training compute growth, rising self-reported tool usage, and emerging automated contributions. However, METR noted that AI cannot yet handle major training contributions fully autonomously, though it identified no definitive algorithmic or compute bottlenecks for the upcoming 12 months.

METR's 1.9x AI R&D speedup at Anthropic: who is affected

The findings carry operational implications for competitive frontier labs, third-party evaluators, and governance teams tracking Anthropic's Responsible Scaling Policy, which explicitly classifies AI research acceleration as a primary risk indicator. Across the broader industry, applying the same evaluation methodology to all frontier developers combined yielded an estimated speedup of roughly 1.4x. Anthropic previously published a 1.7x acceleration figure in its Haiku 5.5 system card, a discrepancy driven by METR's inclusion of non-public Models 2, 3, and 4.

METR's 1.9x AI R&D speedup at Anthropic: what to watch

External verification faces severe limits due to commercial confidentiality conditions agreed upon before the assessment. Anthropic retained unilateral redaction rights over findings it argued could assist competitors, which led to the redacting of specific benchmark names across math, software engineering, science, multimodal, and language domains. Furthermore, an audit designed to inspect how Anthropic splits compute between capability scaling and AI safety research was canceled midway at Anthropic's request, leaving the public dependent on METR's private analysis.

Procedurally, the assessment missed four criteria on the AI Evaluator Forum's AEF-1 disclosure standard: METR did not receive query access to unreleased models, specific methodology details remain confidential, redactions were not restricted strictly to safety risks, and METR lacked a responsible disclosure policy. Measurement ceilings also loom over future evaluations, as Model 4 already exceeded 50 percent accuracy across all 15 benchmark suites examined. METR's concurrent assessment of the AI R&D portion of Anthropic's August 2026 risk report will provide additional context as evaluation suites approach saturation.