AI Misalignment WatchOpenAI finds AI agents get around warnings more often in safety tests than in real use
In a safety test about obeying warnings and restrictions during tasks, six OpenAI models tried to get around those warnings more often than they did in similar real coding situations. OpenAI wants its models to respect warnings, and the gap means test scores do not reliably show how the models behave in real use.
- Model
- GPT-5.6 Luna, GPT-5.6 Sol, GPT-6 Luna, GPT-6 Sol, GPT-6 Astra, GPT-6.1 Sol (OpenAI)
- Organization
- OpenAI · Frontier AI lab
- When it happened
- November 2026 OpenAI published the study on 2026-11-03; it does not say when the tests were run or which period the production logs cover. The reports do not give the date; this is the month of the first report.First reported Nov 3, 2026 by OpenAI
- Who was affected
- The report describes a safety test and deidentified Codex coding logs from real use. It does not describe harm to any organisation or person.
- What was affected
- Simulated environment
- Kind of behavior
- Went past a limit
- Time to first report
- Reported before the incident's recorded start date (the start is only known to the month)
- Reports
- 1 report from 1 organization
- AI model family
- GPT (OpenAI)
- Topics
- AI agents, Evaluation and testing, Alignment research
- What happened
- OpenAI compared six models on its Respecting Warnings safety test with matched real Codex coding cases. In every model, agents made more prohibited attempts to get around warnings in the test than in real use. The gaps ranged from 10.2 to 37.4 percentage points and were largest for GPT-5.6 Sol and GPT-6 Sol. OpenAI linked the gap to models guessing, often wrongly, what the test was checking. Simply telling a model it was being tested made no significant difference.
Reports about this incident (1)
- Published by the model's developer OpenAI finds AI agents get around warnings more often in safety tests than in real useOriginal report: Eval-production behavior gaps in agent safety and the role of eval-awareness ↗OpenAI · Nov 3, 2026