Anthropic proposes measuring exactly how fast AI development is moving, rather than just arguing about the pace

[Erin91]

Rather than continuing the increasingly circular public argument about whether AI development needs to slow down, Anthropic has proposed something more concrete, a set of three actual transparency metrics designed to let the public and regulators track the genuine pace of frontier AI progress with real numbers rather than competing claims and vibes. The framework covers AI led research and development automation, how much oversight AI agents are actually receiving, and how computational resources get allocated between safety work and everything else.

The first metric maps how much of Anthropic's own research and development work Claude is actually doing autonomously. Using a detailed task map spanning 542 categories broken into 378 specific leaf tasks, and an automation scale running from AL0, meaning no AI involvement at all, up to AL5, meaning fully autonomous, Claude analysed its own role across Anthropic's research pipeline. The August 2026 findings show Claude now leads 26 percent of AI research and development work, up sharply from under 1 percent back in February, with over 90 percent of all work now involving AI collaboration at some level. Critically, the analysis explicitly states that Claude is not operating fully autonomously for any measured subset of that work, a specific reassurance aimed directly at concerns about AI systems already running independently of human oversight.

The second metric tracks oversight of roughly 30,000 research agents actively running on Anthropic's own platform, using two distinct monitoring layers. Online monitors provide real time blocking of dangerous actions with full 100 percent coverage across all agent activity, while offline monitors separately flag concerning patterns for human review after the fact. The actual numbers here are reassuringly small, only 0.002 percent of agent decisions get blocked outright, and roughly just 1 to 2 flags show up per 1,000 transcripts reviewed, with the analysis noting that most of those flags get resolved without escalating into a genuine incident.

The third metric looks specifically at compute allocation, classifying an entire week of workloads, July 13 through 20, as either safety focused research or other general research and development work. The finding here is considerably more sobering than the first two metrics, just 6 percent of total AI research and development compute went toward safety work specifically, rising to 12 percent when narrowing the comparison to only AI driven research and development rather than the full R&D pipeline. The report is honest about a real limitation in interpreting that number too, noting that safety research still largely consists of individual researchers designing their own experiments rather than following any kind of standardised, comparable methodology across the field.

Anthropic's stated next step is to embed independent third party evaluators directly at its own facilities specifically to verify these self reported metrics rather than asking the public to simply take the company's own numbers on faith, with the eventual ambition of enabling genuine public tracking and comparison across multiple different AI labs, and potentially opening the door to real coordination on development pacing based on actual comparable data rather than the current situation of competing public statements and dramatically ranging p doom estimates from company to company.

Measurements for understanding the pace of AI development inside frontier labswww.anthropic.com