Goodfire shares a way to break a language model's settings into simple parts

In short

Goodfire, an AI research company, reports a method for breaking a small language model's internal settings into simple, separate pieces. The team says it used these pieces to understand how part of the model works and to change the model's behavior without retraining it. The report does not describe any model misbehaving.

What happened

Goodfire published a summary of a research paper on May 5, 2026. The authors are from Goodfire, MATS, and independent work. The method is called VPD, short for AdVersarial Parameter Decomposition.

A language model's "parameters" are the huge tables of numbers that determine how it works. Goodfire says these are hard to read. Most earlier work, it says, has looked at the model's activations, which are the values it produces while running. Goodfire wants to study the parameters directly.

The team applied VPD to a language model with 67 million parameters. That is small by current standards. VPD splits the parameters into "subcomponents," which are simple pieces that add back up to the original model. Goodfire says it found about 10,000 of them across 24 weight matrices, and that they often switch on in clear contexts, such as particular kinds of words or sentence structure.

Goodfire reports three results from this work:

• Attention: It says it recovered two behaviors in the model's attention layers, which decide which earlier words the model looks at. One is a "previous-token" behavior. The other routes information at syntax boundaries. Goodfire says these were spread across several attention heads.
• Editing: It changed one subcomponent that recognizes the eyes of emoticons such as ;) and :(. After the edit, the model predicted a shocked face for all emoticons. No training was needed. Goodfire says the edit had about as few side effects as fine-tuning methods, though it still performed worse.
• Subnetworks: It traced the small set of pieces the model uses to predict "her" in "the princess lost her crown."

What was done about it

Nothing needed fixing, because the report describes no problem. Goodfire says it wants to try the method on larger models. It also says the technique will probably need further refinement before that works.

Why it matters

Goodfire says its aim is to understand neural networks well enough to design them on purpose, with more of the properties people want and fewer of those they do not. It calls VPD a step toward that goal. The source does not tie this work to a specific safety or misalignment problem. All results are from one small model, so how well the method works on larger ones is untested. Readers can find the full details in the original report.