Goodfire essay argues neural networks are easier to interpret than thought

In short

Goodfire, an AI research company, has published an informal essay by researcher Nick Cammarata. He argues that neural networks are easier to understand than many researchers have believed. This is a working hypothesis, not a report of an AI model misbehaving.

What happened

Goodfire published the essay, which it calls "an informal research essay and working hypothesis." Cammarata says he spent many months studying neurons and circuits in image and language models. A neuron here is a small unit inside a network that responds to certain patterns in its input.

For years, many researchers thought some neurons were hopelessly tangled. The leading explanation was superposition, the idea that a network that runs out of room packs unrelated things into one neuron. Cammarata says he helped promote that idea. He now believes it was largely wrong.

He reports that a famous neuron once read as "cats and cars" is really a detector for V shapes. He says a well-known circuit once read as a car hidden among dog neurons actually reflects shared body pose. He also says the late layers of an early image model, InceptionV1, looked tangled because of ordinary design flaws and weak tools. With modern changes to the design, he reports that 99–100% of neurons in its last layer are easy to interpret.

For language models, he describes preliminary audits of a model called Qwen3-32B. He estimates that roughly 90% of neuron poles have one main job, about 7% have two related jobs, and only 2–4% are not yet understood. He says similar results hold in models up to trillions of parameters, and a companion paper will give more detail.

He adds a caveat. The exact percentages depend on where researchers draw the line for what counts as "one thing."

What was done about it

Nothing needed fixing, because no incident is described. Goodfire says a technical companion paper will cover the language model experiments in more detail. Cammarata says the final step of his argument, that this approach might be most of what interpretability needs, is "highly speculative."

Why it matters

The essay says that if neurons can be read directly, researchers could study a network's own parts instead of first untangling it. Cammarata argues this could make interpretability, the work of reading what is happening inside a model, more tractable than feared. He suggests that behaviors such as deception may rest on small internal parts that are hard to know without looking closely. The claims come from one researcher's work, and the essay describes them as a hypothesis. Samples are described as small in places, and the language model audits are called preliminary. You can read the original report for the full argument.