Apollo Research calls for a 'Science of Evals' to make AI tests more reliable

In short

Apollo Research, an AI safety research group, published a post arguing that tests of AI models, known as "evals," are not yet rigorous enough to support high-stakes decisions. It calls for a "Science of Evals" built on shared standards and collaboration. This is a position post, not a report of an incident.

What happened

Apollo Research published the post on 22 January 2024. It is a call for action. Apollo says it is "not a solution outline."

Apollo says evals are used to reduce uncertainty about how AI models behave. It notes that they underpin many Responsible Scaling Policies, which are company commitments to adjust safety measures as models grow more capable. It also says future laws might link risk limits to specific evals.

The group says evals are a young field and that current ones are not yet resistant to close scrutiny. Its main example is that language models can be very sensitive to how a prompt is worded or formatted. It cites a study finding accuracy differences of up to 76 points from subtle formatting changes. It also cites an Anthropic post saying simple changes, such as switching "(A)" to "(1)", can shift accuracy by about 5 percentage points. Apollo concludes that evals are currently "much more an art than a science."

Apollo also says it is hard to know whether a negative result is meaningful, because better prompting methods keep raising what models can do. It points to rival results on one benchmark, MMLU, changing within days.

The post compares this with shoes and airplanes, which go through extensive testing. It describes how a field matures, from early exploration to formal standards, and lists questions a mature field should answer. These include what an eval measures, how widely it covers possible inputs, and whether its results can be repeated.

What was done about it

Nothing here is a response to an incident. Apollo recommends funding for this research from bodies such as the National Science Foundation, NIST and AI safety institutes, and from large AI labs. It also suggests early collaboration between researchers, industry and lawmakers, and borrowing methods from other sciences, such as statistical significance thresholds. Apollo says it wants to work with academics, industry, auditors, lawmakers and funders, and invites them to get in touch.

Why it matters

Apollo says evals are likely to be an important part of AI safety and are already tied to safety-related decisions. It argues that, for them to be used for high-stakes choices, people need high trust in how they are run. It warns that this matters especially if a decision not to release an AI system carries large financial costs for a company and is challenged in court. The post is Apollo's view, and it describes its list of related research as not exhaustive. Read more in the original report.