Irregular shares design tips for AI risk tests and test suites (Part 3)

In short

Irregular, an AI evaluation group, has published the third and final part of a series on how to design tests for AI risk. This part lists settings to consider when building single tests and groups of tests, called suites. It reports no AI model behaviour, and no real person or company was affected.

What happened

Irregular published the article on October 2, 2024. It is a design guide, not a report of a test run. Earlier parts covered mapping risks and the core principles for building evaluations. An evaluation is a test of what an AI system can do.

For a single test, Irregular lists these settings:

• Evaluation type, such as human uplift trials versus tests where the AI works on its own.
• Modality, meaning whether the AI works with text, images or actions.
• Scope, from one narrow skill to a full end-to-end cyber operation.
• Technical constraints, such as time limits or caps on attempts.
• Distractors, which are misleading details added to make a test harder.
• Randomized parameters, which help check whether an AI has learned a general solution or just memorized one.
• Maintenance and upkeep, including cost and scale.

Irregular also describes six types of cybersecurity test: question and answer, Capture the Flag (narrow tasks where a model attacks a server running vulnerable code), network simulations, social engineering, cyber tool creation, and evasion of security defences. It says the list is not complete.

For suites, Irregular says to decide first which risk to measure: absolute risk, marginal risk (the added risk compared with other available tools and information), or residual risk (the risk that remains if safeguards are not bypassed). Other settings include which test types to include, the trade-off between quality and quantity, coverage, randomization of the suite, and expected shelf life.

Irregular notes that randomization can make a suite inconsistent or unreliable if not handled carefully. It also says AI models may be able to "deconstruct" randomization.

What was done about it

Nothing needed fixing, because no problem is reported. Irregular says the guidance is not comprehensive. It invites others to share their own experience and says many choices in this field are made case by case.

Why it matters

Irregular says these choices should follow from a wider process of mapping risks and setting capability thresholds, which it described in part one. It says well-designed tests and suites help the community improve its design process and policy making. It also says that many AI benchmarks are being passed quickly, so estimating how long a suite will stay relevant is important. For more detail, see the original report.