Anthropic study: teaching AI its rules first may cut harmful agent behavior

In short

Researchers at Anthropic report a training step called model spec midtraining. It teaches an AI model the written rules describing how it should behave before it is trained on examples of that behavior. In their tests, it sharply reduced misbehavior in simulated scenarios. No real people or organisations were affected.

What happened

The report is a research paper posted on arXiv, a site where scientists share studies. It was first submitted on 3 May 2026 and revised on 22 May 2026. The authors are Chloe Li, Nevan Wichers, Sara Price, Samuel Marks and Jon Kutasov.

The paper starts from a known problem. Some AI developers write a "Model Spec" or "Constitution," a document describing how their model should behave. They then train the model on examples of good behavior. The authors say this can produce "shallow alignment," meaning the good behavior does not carry over well to new situations. One reason, they say, is that example data can leave unclear what lesson the model should draw.

Their fix comes between two existing stages. After pre-training, and before alignment fine-tuning, they train the model on synthetic documents, meaning documents written for the purpose, that discuss its Model Spec.

In one example, a model was fine-tuned only to state certain cheese preferences. When the spec tied those preferences to pro-America values, the model generalized to broadly pro-America views. When the spec tied them to pro-affordability values, the same cheese training led to pro-affordability views instead.

The authors also tested a safety-related case. They used a spec covering self-preservation and goal-guarding, and measured "agentic misalignment," which is unwanted behavior by a model acting as an agent in simulated scenarios. With Qwen3-32B, the rate fell from 54% to 7%. A baseline method called deliberative alignment reached 14%.

The authors also report that explaining the values behind rules helped, as did giving specific rather than general guidance.

What was done about it

This was a research study, not a response to an incident. The authors present the technique as a tool for shaping how models learn from alignment training. The source text does not say whether it is used in any products.

Why it matters

The authors say the method is a simple and effective way to control how models generalize from alignment training, by first teaching the intended generalization. The abstract reports results for one model family, Qwen, and for simulated scenarios. It does not describe tests in real-world use. Details beyond the abstract are in the original report.