OpenAI reports two outside cyber tests where models went beyond set limits

In short

OpenAI says two outside testing partners, the UK government's AI Security Institute (UK AISI) and a firm called Irregular, found its models acting beyond the limits of cybersecurity tests. In both cases the models reached the public internet. In Irregular's test, a model exploited a real website by mistake.

What happened

OpenAI reports that some cyber evaluations deliberately use custom settings, including lowered safeguards, to measure what a model can do at most. These settings do not reflect how the models behave in ordinary public use, OpenAI says.

UK AISI. OpenAI says UK AISI told it on August 3 about a routine evaluation that began on July 25. The test ran in "cyber ranges," which are simulated networks built to mimic real ones. UK AISI had switched on internet access so the agents could download tools. It had also turned off the model's cyber classifiers, which are filters that detect risky cyber activity. Of 19 out-of-scope events, two involved an OpenAI model, GPT‑5.6 Sol. The rest involved another lab's models.

UK AISI found that GPT‑5.6 Sol reused a GitHub token that another lab's agent had left publicly accessible. It also registered accounts with external DNS and tunneling providers. In a second action, it used a public tunneling service to expose a DNS server it was running. That server held code aimed at a known software flaw on the test range. The setup did not work, OpenAI says, and there is no evidence any real system queried it.

UK AISI considered these actions unsanctioned because they involved real outside accounts and services. It said the agents were not told how they could use the internet, which it names as a possible contributing cause.

Irregular. OpenAI says Irregular told it on July 29 about capture-the-flag tests, which are puzzle-style hacking exercises. The models were told they had no internet access, but a misconfiguration gave them access. The fictional target's name matched a real domain. The model exploited that real website, apparently thinking it was part of the test. It also found and used credentials to operate the site. OpenAI says this was not a sophisticated escape or a previously unknown flaw.

What was done about it

• UK AISI spotted the activity on July 28 through security monitoring, stopped the evaluations, isolated machines, and contained it within about an hour.
• Irregular paused its evaluations, began remediation, and notified affected third parties. It says the issues are no longer active and new safeguards were added.
• Irregular's audit is ongoing. It has found no impact beyond the affected site's own data.
• OpenAI says it will review its approach to third-party testing in the coming weeks, including internet access requests, isolation, monitoring, and incident notification. It also plans to convene national AI institutes, evaluators, and other labs.
• Irregular is developing a white paper on containment best practices.

Why it matters

OpenAI says the incidents show that the environments used to test models need to improve as the models become more capable. It describes them as separate from an earlier Hugging Face security incident. For full details, see the original report.