Anthropic’s AI is improving itself and could suddenly become ‘extreme’ and ‘risky’, major report warns

Anthropic 's artificial intelligence models are accelerating their own development, a new study has found, raising concerns about out-of-control AI .

An independent assessment by an AI safety monitoring group found that the Claude maker's AI progress is now twice as quick as it was a year ago.

The findings, which were shared by the non-profit Model Evaluation and Threat Research (METR), come amid calls from Anthropic CEO Dario Amodei to slow down the pace of AI development.

In a September essay, titled 'We Must Pace the Frontier', the tech boss proposed a three-point plan to develop artificial intelligence models safely.

"Since roughly this summer, AI has been advancing drastically faster, driven primarily by AI's growing ability to build the next generation of AI," he wrote.

"This dynamic is called recursive self-improvement... Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all."

OpenAI CEO Sam Altman said at the time that he agreed that "we need to pace the frontier", adding that Mr Amodei's proposal for independent evaluators was "a great idea".

Anthropic invited the researchers from METR to review its internal data and conduct evaluations of its models' capabilities.

The report, which was published on Thursday, noted that Anthropic had effectively achieved two years of progress in benchmark progress over the past year.

The accelerating development was a result of recursive self-improvement on the part of the AI models, rather than growth in the amount of computing power used to train models, the researchers said.

"If AI systems started to significantly accelerate further AI development, this could lead to a sudden emergence of extreme and risky capabilities," the report stated.

The researchers also noted that humans are still needed to oversee the most critical parts of AI development, though this could soon change.

"We understand that AI is not yet able to make highly significant and fully autonomous contributions to the most important parts of the model training process," they wrote.

"However, researchers described significant 'signs of life' implying such contributions could occur soon, although with substantial variation in maturity depending on the area."