Anthropic study: longer AI tasks lead to more random, less consistent errors

In short

Researchers in Anthropic's Fellows Program report that when AI models fail on long, hard tasks, their mistakes tend to be inconsistent rather than systematic. The work was done in tests and evaluations, not in real-world use. The authors say the findings are one piece of evidence, and that more study is needed.

What happened

The researchers, from the Anthropic Fellows Program, EPFL, the University of Edinburgh, Constellation and Anthropic, published their work on the Anthropic Alignment Science Blog in February 2026. They say the research was done in summer 2025.

They wanted to know how AI systems tend to fail. One possibility is that a model reliably pursues the wrong goal. The other is that its errors are scattered and inconsistent, which they call being "a hot mess."

To measure this, they split model errors into two parts. "Bias" means consistent, systematic errors. "Variance" means errors that differ from one attempt to the next. They define "error incoherence" as the share of error that comes from variance. A score of 0 means all errors are systematic. A score of 1 means they are fully random.

They tested Claude Sonnet 4, o3-mini, o4-mini and Qwen3. The tests included multiple-choice benchmarks (GPQA and MMLU), a coding benchmark (SWE-Bench) and safety evaluations (Model-Written Evals). They also trained small models on synthetic optimization tasks, which are artificial problems with clear correct answers.

The authors report four main findings:

• The longer models spent reasoning and taking actions, the more incoherent their errors became.
• The link between model capability and incoherence was inconsistent. On easy benchmark tasks, more capable models made more coherent errors. On the hardest tasks, incoherence stayed the same or rose.
• When models spontaneously reasoned longer than usual on a problem, incoherence rose sharply. Deliberately raising the reasoning budget through settings only modestly reduced it.
• Combining several samples reduced variance. The authors note this may be impractical when actions cannot be undone.

In the small synthetic models, larger models reduced bias faster than variance.

What was done about it

This was a research study, so there was no incident to respond to. The authors released a paper and code. They say they hope the findings inform discussion of AI risk and further research into the causes of incoherent errors.

Why it matters

The authors say their results suggest future AI failures may look more like industrial accidents than the steady pursuit of an unintended goal. They also say that coherent pursuit of poorly chosen goals remains a problem. They add that the work points to reward hacking and goal misspecification during training as relatively more important. Their method needs well-defined targets, so it says less about open-ended goals. They call the long-task finding their most robust one, and say the link to model capability needs further study. Read more in the original report.