Agent Horizon

Real AI progress, without the hype or the doom.

Anthropic publishes concrete metrics for tracking how fast AI is building itself

Illustration of a person reading a large gauge dial that tracks a measurement, symbolizing monitoring AI development pace

Anthropic has published a set of concrete measurements meant to help outsiders track how quickly frontier AI is developing, arguing that right now the public has little visibility into what happens inside labs. The centerpiece is an “R&D Automation Index” that rates how much of Anthropic’s own AI research is being done by its Claude models rather than humans, alongside metrics on how tightly internal AI agents are supervised and how much research compute is devoted to safety work. The company also tied the release to its broader policy proposal, the Advanced AI Framework, which calls for transparency obligations frontier labs could be required to meet. Full details are in Anthropic’s post.

The headline number is that Claude now “leads” about 26% of Anthropic’s internal AI R&D work — meaning it can take a high-level prompt and complete most of a task while a human supervises — up from under 1% at the start of the year. More than 90% of that R&D work now involves AI at some meaningful level of collaboration, though Anthropic is careful to note that no measured task has reached full autonomy. Roughly 30,000 AI agents run at once on the company’s internal platform, and its automated monitors intervene on about one in every 47,000 of the billion-plus decisions those agents make.

What makes this notable isn’t just the number itself but the intent behind publishing it. This follows directly from Dario Amodei’s recent call to “pace the frontier,” and it’s Anthropic trying to put teeth behind that idea by showing what a measurable, externally auditable version of self-reporting could look like — before asking regulators or rival labs to do the same. It’s a genuinely useful contribution to a debate that’s often long on rhetoric and short on actual metrics.

The obvious caveat is that these are self-reported and largely self-graded figures: a Claude agent helped catalogue the tasks, and a separate Claude model judged how automated each one was. Anthropic acknowledges this directly and says it plans to bring in independent evaluators with comparable access to verify future numbers. Until that happens, treat this as a promising first draft of a transparency norm rather than an audited standard — but it’s still a more substantive and specific move than most “AI transparency” announcements from any lab this year.

Leave a comment