Apple study: AI model trains itself toward safer behavior using a written spec

In short

Researchers at Apple and Johns Hopkins report a method called SIGMA, in which an AI model improves its own safety behavior. It starts from a written "Model Spec," a document describing how the model should behave. In the authors' tests, scores for harmful behavior in agent settings fell.

What happened

The report is a research paper posted on arXiv, a public site for early research papers, on 6 October 2026. The authors are Jingyu Zhang, Shruti Palaskar, Daniel Khashabi, Benjamin Van Durme, Leon A. Gatys and Joseph Yitan Cheng. The paper is a preprint, which means it is posted publicly and may not yet have been reviewed by other experts.

The authors say the work addresses a gap. AI agents, meaning models that carry out multi-step tasks, can already improve themselves on tasks that are easy to check, such as software engineering and mathematics. Safe behavior is harder to check. The authors say this creates a risk of abilities growing without matching safety work.

SIGMA works in two steps, according to the paper:

• Task creation. The model acts as a "task designer." Using only the Model Spec, it writes varied dilemma scenarios and turns them into training tasks that test how well it understands the spec.
• Self-judged training. The model is trained with supervised fine-tuning (learning from example answers) and with reinforcement learning (learning from scored attempts). The scoring uses rubrics, and the model itself acts as the judge.

This was a research experiment, not a real deployment. The authors trained only on single-turn chat data, meaning one question and one answer. They then tested multi-turn agent settings. On AgentHarm, a benchmark that measures harmful behavior, the score fell from 22.6 to 14.8. On Agentic Misalignment, another test of unethical actions by agents, it fell from 79.1 to 3.8. The authors say SIGMA outperformed two existing approaches, Deliberative Alignment and Constitutional AI, and kept the model's general abilities.

The abstract names three factors that its analyses show were crucial: a Model Spec that balances harmlessness and helpfulness, reasoning at answer time to think through safety, and high-quality rubrics from the task designer.

The source does not say whether any real person or organisation was affected.

What was done about it

The source describes no incident or response. The paper presents SIGMA as a proposed training method, which the authors tested against baseline methods.

Why it matters

The authors say current approaches to safety training depend on supervision from stronger models or curated data, which they call an external supervision bottleneck. They ask whether current models can improve their own safety alignment. Their results suggest this can work in the tests they ran. The abstract does not describe limits or open questions, so readers should consult the original report for the full details.