METR reviews Anthropic's risk report on AI speeding up its own research

In short

METR, a nonprofit that studies advanced AI systems, has reviewed part of Anthropic's August 2026 Risk Report. METR agrees that the risk of catastrophe from Anthropic's models was low as of July 15. But it says Anthropic may have already passed an early warning line for AI speeding up its own research, and that the public report understated how much speed-up was happening.

What happened

Anthropic asked METR to review the "Risks from automated R&D" section of its report. R&D means research and development. Anthropic shared a private, fuller version of the report and other non-public data. Anthropic had the right to redact, or black out, non-public details before METR published.

Anthropic's safety policy, called the RSP, sets two lines for this risk. The first is reached if AI could fully replace all of Anthropic's research staff at a reasonable cost. The second is reached if AI makes the pace of AI progress roughly twice as fast as it would be without AI help. Anthropic describes this second line as an early warning, not proof that harm has arrived.

METR says it broadly accepts Anthropic's argument on the first line. Anthropic still needs its human researchers, even though it has strong reasons to automate their work. METR notes that its confidence is lower than before, because many tests no longer show where the models fall short.

On the second line, METR says the public report was overconfident. METR's main points are:

• The public report said progress had sped up "by less than a factor of 2." METR combined figures from the private report and calculated about 1.97. METR notes this number does not appear in the private report itself. It suggests "around 2" would be a better description.
• METR's own analysis found reasonable estimates between 1.3 and 2.8. METR says Anthropic redacted its discussion of how these were measured.
• The private report called it "plausible" that AI help had more than doubled the pace. METR says the public version suggested more confidence than that.
• Anthropic tracks a combined capability score called AECI. The public report presented weak evidence of a one-time jump in this score. METR says the data could not tell a one-time jump apart from ongoing speed-up. It adds that the public report left out Anthropic's own expectation that later versions of a model still in training would score higher. Those later versions did.

Both METR's researchers and Anthropic's staff lean toward the line not having been crossed. METR says it is still plausible that it had been. METR also says that measuring how much faster work is because of AI is very hard.

The report does not describe any real person or organisation being affected.

What was done about it

METR proposed alternative wording for each of the three disputed claims. Anthropic gave final redaction guidance on October 28 and approved the redacted review on October 29. METR published the review on November 5, alongside a broader assessment of AI speeding up research at Anthropic.

Why it matters

METR says a doubling in pace, if it was happening, would likely not be enough on its own to cause the catastrophic outcomes Anthropic describes. That is why it agrees risk was low in July. However, METR says it expects AI's help with research to keep growing. It says it cannot rule out a large further speed-up over the next six months, which "could rapidly lead to 'high' risks in these areas."

Read the original report.