Anthropic data highlights AI doomer concerns

Investing.com -- Anthropic has released new measurements showing how much of its AI research is now being performed by AI itself, providing fresh data for a central question in the AI-doomer debate: whether increasingly capable AI could eventually accelerate the development of even more capable systems.

The company said Claude now “leads” 26% of Anthropic’s AI R&D work, up from less than 1% in February. Anthropic defines “leads” as completing most of a task end-to-end from a high-level prompt, with a human supervising. More than 90% of AI R&D is performed at or above the level where AI “collaborates” with humans.

The significance is less the 26% figure itself than the potential feedback loop: AI performs AI research, that research produces better AI, and better AI performs more AI research. This is a key component of the recursive-self-improvement scenario that has fueled concerns about the long-term risks of advanced AI.

Anthropic said it had roughly 30,000 AI agents simultaneously conducting research and engineering work on its most-used internal platform in August. More than 1 billion agent decisions were analyzed during the month, with about 0.002% blocked by online monitoring.

The company has not reached the most important threshold for the recursive-self-improvement scenario: fully autonomous AI R&D. Claude is not yet operating autonomously for any measured portion of Anthropic’s R&D, meaning humans remain in the loop.

Still, the pace of change is notable. The share of AI R&D classified as AI-led rose from below 1% to 26% in just six months.

Anthropic also found that about 12% of compute used for AI-driven AI R&D went toward safety work during one week in July, although the company cautioned that compute is an imperfect measure of safety investment.

The measurements do not show that recursive self-improvement has arrived. They do, however, provide a way to track how quickly AI is taking over the work of building future AI — a trend that could become increasingly important if the human role in the development loop continues to shrink.

You May Also Be Interested In

• Huang and Altman to attend Trump-Xi state dinner - ABC
• Goldman Sachs stock falls after CEO warns of softer trading
• The Fed is hiking. Why isn't UBS worried about stocks?