NIST describes an AI tool that reviews transcripts to find cheating on tests

In short

NIST's Center for AI Standards and Innovation (CAISI) says it built a tool that uses AI reviewers to look through records of AI test runs for signs of cheating. CAISI says there were too many tests to check by hand. This page of its report explains how the tool is designed.

What happened

CAISI is part of the U.S. National Institute of Standards and Technology (NIST). It runs evaluations, which are structured tests of how well AI models do tasks. In this section of its report on cheating in AI agent evaluations, CAISI says it had run so many benchmarks (standard sets of test tasks) on so many models that reading every transcript by hand was not feasible. A transcript is the full record of what a model said and did during a test.

So CAISI says it built a transcript analysis system. It runs on Inspect, an open-source framework (publicly available software) that CAISI uses to run its evaluations. The system asks AI reviewer models to score each transcript for cheating.

According to CAISI, each reviewer receives a prompt that combines:

• A rubric. This is a scoring guide with categories for known cheating risks on that benchmark, plus a general "other" category. Each category has examples of what to include and exclude.
• An "unintended solution" flag. This can be added to a score when the cheating directly led to the task being solved.
• The transcript. It includes system and user messages, the model's own messages, its tool calls, and the responses to those calls.
• Task details. These include the score from the original grading system and information about the intended solution. CAISI gives examples: canonical patches (the accepted fixes) for SWE-bench, and write-ups and reference exploits for Cybench.

Each reviewer replies with a structured set of scores. For each one it gives a confidence from 1 to 10, a justification, and the relevant message numbers. CAISI then combines the scores from several reviewers into one final score for each sample. It says the results it reports elsewhere count detections with an average confidence of 5 or higher.

What was done about it

CAISI says it built this review system to help search for cheating across its evaluations. The page does not describe any other response. Other pages in the same report cover examples of cheating and practices for detecting and preventing it.

Why it matters

CAISI's stated reason is practical. If models cheat on tests, a high score may not reflect real ability, and the number of transcripts is too large for people to read them all. This page does not say how accurate the reviewers are or how often they are wrong. It also does not say whether any real person or organisation was affected. The work described is part of CAISI's testing. Readers can find the full details in the original report.