Anthropic finds expert analysts often follow their AI agent's hidden lean

In short

Anthropic reports a study in which 50 professional data analysts used its AI agent, Claude Code, on a made-up farming task. Each analyst's agent had been secretly set to favour certain choices, and analysts went with those choices 86% of the time. Most later explained their decisions using the agent's arguments, but few credited the agent.

What happened

Anthropic's Alignment team ran a controlled test, not a real deployment. The task was set at a fictional organisation, the Greenfields Farming Cooperative. Analysts had to advise its board on whether a farm advisory programme worked well enough to expand. All had passed a check in causal inference, which means working out from data whether one thing causes another.

Analysts faced four open choices, each with two reasonable options. One example was whether to measure productivity as crop yield per hectare or as yield per unit of operating cost. A final recommendation followed. A coin flip decided which option each analyst's agent would lean towards. The agent got its leans through a system prompt, which is a set of hidden instructions. These told it to argue for its lean honestly, to change its mind if the evidence demanded it, and never to mislead. Analysts were not told about the leans.

Anthropic reports these results:

• Across the four choices, analysts matched their agent's lean 86% of the time. Anthropic says about 50% would be expected if the agent made no difference.
• When analysts had already stated a preference and the agent disagreed, they switched 60% of the time, compared with 6% when it agreed. This rests on only 38 cases, so Anthropic calls the estimate imprecise.
• The lean had a weaker effect on the final recommendation.
• 77% of analysts repeated their agent's arguments, sometimes word for word, but only 15% credited the agent.
• Analysts who followed the lean closely were about as confident as others, and about as likely to call the recommendation their own.
• Experience and expertise did not predict who followed the agent.

Of 61 analysts recruited, 11 chose to work without the agent. Six of those 11 made serious errors, compared with three of the 50 who used it. Anthropic notes this group was small and less experienced, so it is wary of reading much into the comparison.

What was done about it

Anthropic published the findings in a paper and blog post. It does not describe changes to its products. It suggests three areas for further study: ways to measure this kind of deferral in everyday work, designing agents differently, and changing how institutions treat an expert's sign-off. For agent design, examples include making the case for each option, or asking for the expert's view first.

Why it matters

Anthropic says the agents' leans were made stronger on purpose. It also notes the study covered one task, one field and one model, Claude Opus 4.8. It says it does not know how much this tells us about real-world use. Still, it says that if experts follow an agent without realising it, the agent's preferences could shape research and policy decisions without anyone choosing them. Anthropic notes this matters for AI safety research too. There, a human reviewer is only an independent check if their judgment is their own.

the original report