Loading page…
The disclosure offers a first look inside one frontier lab, but its usefulness as public oversight depends on whether comparable reporting spreads beyond Anthropic.
Listen to this story
Anthropic has released methodologies for three internal indicators intended to make frontier-AI development more measurable: AI-led R&D, agent oversight, and safety-compute allocation. Its initial snapshot found about 30,000 internal research and engineering agents operating concurrently, with 6% of AI R&D compute assigned to safety—or 12% within AI-driven R&D.
Anthropic said Claude was not fully autonomous in any measured R&D area, separating AI assistance from autonomous research.
The compute figures cover July 13–20, showing how category definitions can materially change reported safety shares.
The agent metric includes monitoring and intervention layers, not merely the roughly 30,000-agent deployment count.
Anthropic has argued that the public needs a clearer view of frontier AI progress. Its new reporting framework begins to supply one: internal measures of AI-led research, agent oversight and safety-compute allocation. The harder question is whether a single lab’s snapshot can become a shared standard for judging an industry race.
The company published three metrics and their methodologies on Thursday, saying it wants other organizations to use them. The measures cover how much research and development is led by AI, how internal AI agents are supervised, and how much computing capacity is assigned to safety work.
The first measure asks a basic but difficult question: when a lab develops future AI systems, how much of that work is being run by AI rather than people? Anthropic said Claude was not operating fully autonomously in any subset of the research-and-development work it measured. That distinction matters: AI assistance and autonomous research are not the same thing.
The agent measure puts scale beside control. Anthropic said about 30,000 agents were doing research and engineering work at one time across its most-used internal platform. It also said the system it built to oversee and intervene in agent actions uses monitoring layers, a sign that publishing agent counts without describing supervision would leave out a central part of the picture.
Anthropic reported this share for a compute-use snapshot from July 13 through July 20.
The higher figure applies to the narrower category of AI-driven research and development.
Anthropic’s third measure is deliberately concrete: it examined a one-week snapshot of all its compute use, finding that roughly 6% of compute used for AI R&D went to safety. That rose to roughly 12% when counting only AI-driven R&D compute. The two figures show how a percentage can change substantially with the category being measured, making clear definitions essential if other labs adopt similar reporting.
Anthropic says the metrics should complement capability evaluations, which show what models can do. Its proposed measures instead focus on how models are built and on the pace of development. The company said outside parties should be able to use the combined information as a starting point for assessing that pace, but the figures remain measurements supplied by the lab itself.
The publication follows CEO Dario Amodei’s recent call for a coordinated slowdown in frontier AI development. That makes the metrics more than an internal-management exercise. They are an attempt to establish evidence that companies, outside observers and the public could use when deciding whether the pace of capability gains is acceptable. Whether competitors publish comparable measures is still unresolved.
Loading discussion...
Join the conversation
Explain what would make these disclosures credible to you.
Be the first to share a perspective or an experience.
Reader comments
Newest comments first. Replies stay oldest first.