Anthropic describes Constitutional AI, training a harmless assistant with AI feedback

In short

Anthropic, an AI company, has described a training method it calls Constitutional AI. In the method, an AI model improves itself using a short list of written rules, instead of people labelling harmful outputs. Anthropic says the result is an assistant that is harmless but still explains why it objects to a harmful request.

What happened

This is a research report, not an account of a problem in a real product. Anthropic published it on December 15, 2022. The source is a summary page with the paper's abstract, so it gives only a short outline of the work.

Anthropic says it wants AI systems to help supervise other AI systems as they become more capable. Its experiments tested ways to train a harmless assistant "without any human labels identifying harmful outputs." The only human oversight comes from a list of rules or principles. That list is the "constitution" in the name.

Anthropic says the method has two phases:

• Supervised phase. The team takes answers from an initial model. The model writes critiques of its own answers and then revises them. The original model is then fine-tuned, meaning trained further, on the revised answers.
• Reinforcement learning phase. The team takes answers from the fine-tuned model. Another model judges which of two answers is better. Those judgments are used to train a "preference model," which then acts as the reward signal. Anthropic calls this "RL from AI Feedback," or RLAIF.

Anthropic also says both phases can use chain-of-thought reasoning, where the model writes out its reasoning step by step. It says this improves performance as judged by humans, and it makes the model's decisions more transparent.

What was done about it

Nothing was being corrected here, because the page does not describe a failure. It describes a training method. Anthropic published the paper and a related policy memo. The page does not say whether the method was used in any product.

Why it matters

Anthropic says the approach makes it possible to control AI behaviour "more precisely and with far fewer human labels." It also says the assistant it trained is "non-evasive." That means it engages with harmful questions and explains its objections, rather than simply refusing to answer. The page does not report detailed results, test numbers or limitations. Those would be in the original report and the full paper it links to.