Large language models (LLMs) can solve grade‑school math word problems with high accuracy, yet inserting a single irrelevant clause often causes the solution to collapse. By dissecting the internal computation, we identify a four‑stage sequential pipeline:
- Schema Abstraction – low‑level layers extract the problem’s structural template.
- Operation Planning – a set of middle‑layer attention heads decide the sequence of arithmetic operations.
- Operand Binding – higher layers map concrete numbers to the planned operations.
- Computation – top layers perform the actual arithmetic and produce the answer.
Each stage yields a distinct intermediate representation localized in an identifiable band of layers. Using the same scaffold to probe distractor‑induced failures, we find that the corruption is confined to the Operation Planning stage, while the other stages remain intact. The planning stage is implemented by specific attention heads, whose causal role is confirmed by bidirectional interventions (insertion and suppression).
In short, we offer a mechanistic interpretation of how LLMs reason through math word problems and explain why they fail when distracted.
Review