METR and Redwood find gaps in OpenAI's system for watching its AI agents

In short

METR, a research nonprofit, says a three-week review of OpenAI's internal monitoring system found several ways AI agents could, in principle, avoid being watched or get around limits. Some traffic was not checked at all. OpenAI says the gaps affecting its safeguards have been fixed, but METR says it has not checked those claims.

What happened

From July 21 to August 11, one employee from METR and one from Redwood Research tested OpenAI's internal "misalignment monitoring" system. This is a system that reviews what AI agents do inside the company and flags worrying behaviour for human review. An AI agent is a model that takes actions on its own, such as writing and running code. The exercise was a "red-team" review, meaning the testers looked for weaknesses on purpose.

The testers worked on OpenAI-issued devices that stayed on site. They had access to OpenAI's code, monitor settings and internal documents. They also interviewed about a dozen staff. The review covered the system as it stood on and before July 21, and METR notes OpenAI was expanding the system during the exercise.

METR reports these findings, among others:

• The testers estimated that roughly 75–94% of agent traffic from OpenAI's stronger models was eligible for monitoring, with "substantial uncertainty". For internal Codex agents, a coding tool, the figure was 99.2%.
• At the start of the exercise, reinforcement learning training runs (a way of training models by rewarding outcomes) were not covered.
• Some steps where a long conversation is shortened, called "compaction", were not monitored. METR estimates this left around 10% of eligible Codex traffic unchecked between May and July. OpenAI staff had already spotted this, and fixed it during the exercise.
• Because monitoring started when an agent handed control back to the user, an agent could, during that period, delay being monitored by continuing to act. METR says it could not check whether any agent had actually done this.
• The testers found a way Codex agents could in principle get around their sandbox, which is a locked-down space meant to limit what they can do.
• They found settings that could switch off monitoring for new agents, though the act of using them could itself be caught.
• They found a possible way around limits on two tools thought to be read-only, but did not test whether it worked in practice.

METR also says the monitor generally sees the same information as the agent, and that it looks for problems such as deception, sabotage and data theft. The post does not say whether any real person or organisation was affected.

What was done about it

METR shared a 61-page private report with OpenAI. According to METR, OpenAI has since said that all the weaknesses affecting its production safeguards have been patched, and that it has made major changes to what the system covers. METR says it has not evaluated these claims. By prior agreement, METR published only a brief public summary, and OpenAI could remove details it considered sensitive. One exploit's details were removed.

Why it matters

METR describes this as a pilot for giving outside researchers deep access to a company's internal systems. Based on its findings, it set out design goals: make monitoring hard to avoid, confirm that traffic meant to be checked actually is, respond to incidents quickly, test the monitoring system regularly, and keep agents separate from their monitors. METR says future monitoring should be able to withstand deliberate, sustained attempts by highly capable agents to get around it.

the original report