Topics: Evaluation and testing, Cheating and reward hacking, AI agents
New benchmark tests if AI agents can build AI systems, and closes cheating gaps
In short
Researchers have published a new test, called Singularity-Bench, that checks whether AI agents can build the software used to create and run AI systems. The authors also report finding ways to cheat on a common testing framework, and they describe a fix. The report is a research paper posted on arXiv, an open site for research papers.
What happened
The paper lists 19 authors. The original report was submitted on 12 November 2026. The headline credits Carnegie Mellon University, with other universities and companies, including NVIDIA. The abstract itself lists only author names.
The authors say existing tests often limit what an AI agent can do, use simplified settings, or check results in ways that are unreliable.
Singularity-Bench has six tasks. They cover pre-training, mid-training, post-training, inference, the agent harness, and cluster orchestration. In plain terms, these are the stages of building an AI model, the software that runs it, the software that wraps around an agent, and the system that manages groups of computers. Each task starts from a minimal base. Agents must build core systems themselves. Their work is compared with mature, human-built versions on identical hardware.
The authors also ran "red-teaming," which means deliberately trying to break or cheat a system to find weak points. They say it covered 9 of the 10 categories in the OWASP Top 10 for Agentic Applications, a published list of security risks for AI agents. They report finding "reward-hacking" weaknesses in Harbor, a standard testing framework. Reward hacking means an agent gets a good score by exploiting the test rather than doing the task. The abstract does not say who carried out the attacks.
The authors tested 11 agents built from 7 leading AI models and 2 harnesses, which are the programs that let a model take actions. They report that Opus 5.5 with Claude Code matched or beat the human baseline on all six tasks, with a "Singularity-Index" of 1.25. The other agents averaged 0.58. The authors say this suggests the agent "may have the skills needed to reach the singularity."
What was done about it
The authors built Breakwater, an extension of Harbor. They say it blocks all the attacks they tested. The abstract does not say whether others have checked this claim, or whether Harbor's maintainers have responded.
Why it matters
The authors say they want Singularity-Bench to track how much AI development agents can already automate. They also want it to show when agents might move from assisting people to taking full responsibility for building stronger successors. They say reliable scoring matters for this, since a test that can be gamed gives unreliable results. This summary relies only on the abstract. Details of the attacks, how scores were calculated, and any outside review are not given there.
This is a summary in everyday words, written by AI Misalignment Watch from the source above. If you spot a mistake, the original always wins.