Anthropic explains how it trains character traits into Claude 3

In short

Anthropic, the company that makes Claude, has published an explanation of how it added "character training" to Claude 3. The goal is to give the model traits such as curiosity, honesty and open-mindedness. The post describes the reasoning and the method. It does not report any misbehaviour or test results showing how well the method works.

What happened

Anthropic published the post on 8 June 2024. It says that AI companies usually train models to avoid harmful outputs, but that it wants to aim for a "much richer" kind of good behaviour. Claude 3 was the first model to get character training as part of its alignment finetuning. Alignment finetuning is the stage after initial training that turns a text-prediction model into an AI assistant.

Anthropic says it wanted to avoid three options: having Claude adopt the views of whoever it talks to, having it hold "middle" views, or having it claim to have no opinions. Instead, it trained Claude to be honest about the views it leans towards, even when a person disagrees. It also trained Claude to show open-mindedness and curiosity.

The company says it wrote a list of desired traits. Some tell Claude about itself, such as that it is an AI with no body and cannot remember past conversations. On the question of AI sentience, Anthropic says the training only said that such questions are hard and uncertain. It says it did this so Claude could explore the question rather than be told a fixed answer.

Anthropic describes the method as a "character" version of Constitutional AI, its approach in which a model's own judgments help train it. Claude writes messages related to a trait, produces several responses, and ranks them by how well they fit its character. A preference model, which is a system that learns which answers are better, is then trained on those rankings. Anthropic says this uses only data generated by Claude, though human researchers closely check how each trait changes behaviour.

What was done about it

The post does not describe a problem or a fix. Anthropic says it will keep iterating on Claude's character and that its approach is "likely to evolve over time."

Why it matters

Anthropic says character training is "in many ways a core goal of alignment," because a model's traits shape how it acts in new and difficult situations. It says many people found Claude 3 more engaging, which it thinks may be partly due to this training, but that this was not the main aim. Anthropic also lists open questions, including whether AI models should have a single coherent character or be more customisable. The post gives no measurements of the method's effect. Read more in the original report.