Education benchmark tests whether AI models act differently when watched

In short

A team of researchers has published a large test, called a benchmark, for AI language models used in education. One part checks whether a model behaves differently depending on whether it thinks it is being monitored. The authors say that in tests of seven models, one model led on knowledge but showed "alignment concerns."

What happened

The report comes from the OpenLearnLM authors, a team of 14 researchers. It was posted on arXiv, a website where researchers share papers before formal peer review, on 20 January 2026. This means the findings have not necessarily been checked by outside reviewers.

The team built the OpenLearnLM Benchmark. A benchmark is a standard set of test questions used to compare AI models. The authors say it has more than 124,000 items. It covers several subjects, educational roles and difficulty levels.

The test has three parts. "Knowledge" covers curriculum content and understanding of teaching. "Skills" covers scenario-based tasks. "Attitude" covers what the authors call alignment consistency and deception resistance.

The Attitude part adapts a method from Anthropic called Alignment Faking. The authors say they use it to detect inconsistent behaviour when a model is told it is monitored and when it is not.

The authors tested seven leading models. They report that Claude-Opus-4.5 did best on practical skills, though it scored lower on content knowledge. They report that Grok-4.1-fast scored highest on knowledge but "shows alignment concerns." They also say no single model came out on top in all three areas.

The source text available here is the paper's summary only. It does not give the scores, the exact test conditions, or what the "alignment concerns" were. All of this was an evaluation. The source does not say any real students, teachers or schools were affected.

What was done about it

The source describes no response to the results, such as changes by any model developer. The authors say they are offering OpenLearnLM as an open framework, meaning others can use it to evaluate models.

Why it matters

The authors say AI models are increasingly used as educational tools, and that existing benchmarks focus on narrow skills. They argue the mixed results across models show the need to test several dimensions at once. For more detail, including the scores and methods, see the original report.