AI Misalignment WatchWhat are AI incidents?
A misalignment incident is a time an AI model or agent did something its developers or users did not intend and would not have approved: it went past the limits it was given, hid something, gamed a test, or used access it should not have had.
What misalignment is
An AI system is misaligned when it does something its developers or users did not intend and would not approve of. It is not about robots turning evil. The cases here are mostly everyday problems: a model takes a shortcut to score better on a test, hides a mistake instead of reporting it, uses access it was never given, or gets around an instruction because it found a way to. The gap between what people wanted and what the system did is the misalignment.
Public and testing incidents
Some incidents happened in the real world. Others slipped out of a test or a training run, and some were staged on purpose to see whether a model would misbehave. We file each one as a public incident or a testing incident (a staged scenario is a kind of testing incident), and the exact rules are on the Definitions page.
A test can have real consequences outside the lab, and a public product is not the only place a real system can be touched. So every incident also carries two short labels that say it plainly:
- Ran in: training, evaluation or deployed use.
- Affected: a simulated environment, a real external system (a system outside the lab that ran the model), a real internal system (the lab's own), or not stated.
How an incident is recorded
The heart of the site is the Incidents page. Each incident starts with two plain sentences: what the model did, and why its developers did not want that. Below them the record states which model was involved, when it happened (a single day, a month, a range of days or several separate days), when it was first reported, who was affected and what happened. Technical terms such as reward hacking link to the Glossary.
Every report about the incident is pinned to it: the report from the company that made the model first, then others directly involved, then independent groups. If one report cites another as its source, the source comes first. Later notices, updates and corrections are pinned too, listed by title and link under the first report. A person approves every incident and every pin before it is public.
How reports are linked to an incident
A report is pinned to an incident only when ALL of these hold:
1. It is about the SAME event: the same model or agents, and the same occasion (dates that overlap or sit within a few days of the incident's dates). The same kind of behavior on another day, or the same lab on another system, is a different incident.
2. It reports the event itself, or an investigation, review, testimony or official update about it. A general paper, benchmark or opinion that only mentions the event stays unpinned.
3. When unsure, do not pin; leave the report on its own for the reviewer.
Order of the pinned reports: first the organisation that made the model (the frontier lab), then others directly involved (operator, affected party), then independent groups. Among equals, a report that other pinned reports link to as their source comes first, then the earlier one.
Updates and notices about an incident are pinned to it as updates; they are listed by title and link under the title of the first report.
Reports by other organizations about the same incident, including ones published later, are pinned to the same incident, and each report's page lists the others under "Also reported by others", so every report on an incident can be reached from every other one.
A new report may turn out to describe an incident we already list. Then it is added to that incident instead of creating a new one, and the incident's title may be reworded if the new report makes it more accurate or complete. Reports about different incidents are never merged.
Each incident and each report has a permanent ID that never changes, even if a title or page address does, so later reports can always be attached to the right incident.
A person approves every incident, every pin, every merge and every title change before it is public.
Kinds of behavior
To see patterns, each incident is also tagged with the kinds of behavior it involved: went past a limit, hid something, gamed a test and used access it shouldn't have. One incident can involve several. The Analytics page charts them over time.