Apollo Research tests a method to break AI networks into simple parts

In short

Apollo Research, an AI safety research group, reports a new method for breaking a neural network into small parts. A neural network is the kind of software behind modern AI. Apollo says the method worked on small toy models, but it needed a lot of tuning and some parts still mixed together. Scaling it to larger models remains a challenge.

What happened

Apollo Research published the work on 11 February 2025. The method is called Attribution-based Parameter Decomposition, or APD. Parameters are the numbers, often called settings or weights, that a network learns during training.

APD splits those parameters into "components." Apollo says each component is meant to match one specific mechanism inside the network. As an example, it says the weights that let a network look up the fact "The sky is blue" could form one component. These components do not have to line up with the network's physical layout, such as single neurons, layers or attention heads. One mechanism can be spread across many neurons in several layers.

Apollo says the method trains the components to meet three goals:

• Faithful: the components must add up to the original network's parameters.
• Minimal: as few components as possible should be needed to reproduce the network's output on any single input.
• Simple: each component should be simpler than the whole network.

The tests were run on three toy models. These are small, artificial networks where the correct breakdown is already known. One packed more features into its neurons than it had room for. The other two carried out more computations than they had neurons, with one of them spreading the work across two layers.

Apollo reports that APD was largely able to split these networks into parts matching the individual mechanisms. However, it says the results needed a lot of hyperparameter tuning, which means adjusting the method's settings by hand. It also says some mechanisms were still mixed together. The team suspects this comes from its use of a "top-k" selection step, which picks only the highest-scoring components.

What was done about it

This is a research result, not a problem that needed fixing, and the source does not describe any real person or organisation being affected. Apollo has published the paper. It says challenges remain in scaling APD to models that are not toy models.

Why it matters

Apollo says its results suggest answers to several open problems in mechanistic interpretability, the effort to understand how a network works internally. It lists finding minimal circuits in superposition, giving a firmer definition of "features," and offering a decomposition approach that does not depend on a particular network design. These claims rest on toy models only. The full details are in the original report.