METR proposes how outside assessors could check AI companies' monitoring of agents

In short

METR, a research nonprofit that tests AI systems, has published a proposal for how an outside assessor could check AI companies' monitoring of their own AI agents. The proposal covers five areas, from how much agent activity is watched to what agents could do without being caught. It does not report any AI misbehaviour.

What happened

On 14 October 2026, METR published a blog post on "control assessments." These ask whether a developer's AI agents could cause serious harm without the developer's security and oversight measures stopping them. An agent is an AI system that carries out tasks by taking actions, such as running code or browsing the web.

METR says developers currently rely heavily on monitors to catch harmful behaviour. A monitor here is usually a large language model (LLM), the kind of AI system behind chatbots, that reads what an agent is doing and flags anything concerning. METR says agents often need broad permissions, such as internet access. That makes simpler protections like sandboxing, which means running an agent on a locked-down test computer, less adequate on their own. According to METR, LLM monitors are the main source of evidence inside leading AI developers about AI misbehaviour.

The proposal sets out five questions an assessor would investigate:

• Coverage: what share of internal agent activity is monitored, and what share should be.
• System weaknesses: whether agents could switch off, break or tamper with the monitoring system.
• Evasion: how easily agents could slip misbehaviour past a working monitor. METR also suggests checking whether monitors and agents could quietly cooperate. One method is "honeypots," meaning realistic traps set to see whether a monitor fails to flag an attack.
• Response: whether flagged incidents reach human reviewers quickly, and whether reviewers could be fooled.
• Impact: the worst harm agents could cause without being caught.

To answer these questions, METR says an assessor would need wide access. This includes company code, activity logs, incident reports, staff interviews and internal computer systems. It also includes at least about 100,000 representative records of agent activity, and about 100 records of the most severe internal incidents. METR describes red-teaming, which means deliberately attacking a system to find weaknesses. This includes inserting the assessor's own test attacks into the monitoring pipeline, with the usual precautions used in security testing.

What was done about it

This is a proposal, not an incident report. METR says the post describes the reasoning it has used to evaluate its own internal monitoring system. It says it will update its list of questions as new AI abilities and unintended behaviours appear. No real person or organisation is reported as affected.

METR notes several limits. Assessors must trust that agents have not tampered with the logs they examine. Replacing direct system access with staff interviews would be "a meaningful limitation." Relying on company employees to collect data for the assessor can reduce how independent the check is.

Why it matters

METR says monitoring is only useful if it actually runs on the relevant activity and cannot easily be avoided. It describes LLM monitors as "a critical part of automated oversight at scale." It expects it to become more important to check whether monitors and agents could work together without the developer knowing. The proposal aims to give outside assessors a structured way to gather evidence on these points.

The original report