Apollo Research says safety tests for AI models are not keeping up

In short

Apollo Research, an AI safety research group, says there are too few good safety tests for advanced AI models. It says the tests take a long time to build. This is the group's own view, published in a post. It is not a test of one model.

What happened

Apollo Research published the post on 11 November 2024. It calls the problem the "evals gap." An "eval" is a safety evaluation, a test used to find out what an AI model can do.

The group says it has worked with governments, AI safety institutes and leading AI developers. From those talks, it is concerned that policymakers and companies make two mistakes. They may overestimate how many high-quality evals exist. They may also underestimate how long it takes to build them.

Apollo says many company and government safety frameworks tie decisions to eval results. It names Anthropic's Responsible Scaling Policy, OpenAI's Preparedness Framework, Google DeepMind's Frontier Safety Framework and the EU AI Act. It says some of the evals these frameworks call for do not yet exist, or are at an early stage.

The post also says coverage is low. Apollo reports that people it spoke with agreed there are far fewer evals than needed. Apollo says these conversations were not structured interviews. It says current evals work more like spot checks than thorough assessments.

Building evals is hard, Apollo says. One suite can take a small team several months. As an example, it says a widely used coding test, SWE-Bench, was later found to include problems that were unclear or unsolvable. A corrected version had to be checked by hired software engineers. Apollo says this is not a criticism of the original authors.

Apollo also says it is an unsolved problem to know whether a test has brought out everything a model can do. A model failing a test does not prove it lacks the ability.

What was done about it

Apollo does not describe a response by any developer or government. It calls closing the gap "an urgent priority" and suggests three steps:

Fund the building of evals, not just the running of them. Apollo says running evals is often only about 10% of the total effort and cost.
Fund research into the science of evals.
Change market incentives, because most eval work now rests on voluntary commitments.

Why it matters

Apollo says evals support high-stakes decisions, such as whether and how to release a model. It expects those stakes to rise as models become more capable. It argues that decisions will be made with whatever evals exist at the time, and will not wait for better ones. The post states these as Apollo's views and expectations. Apollo also says the problems can be solved with more resources, people and time.