Chess, with its deterministic environment and complex reasoning requirements, serves as an ideal testbed for probing the internal computation of Transformers. The Maia-3 chess Transformer accepts Elo rating as a conditioning input while keeping its weights fixed, allowing the network’s “skill level” to be varied from 700 to 2500.
We performed exhaustive head‑ablation across every Elo value and observed three key patterns:
- Increasing Elo monotonically pushes the causal center of mass of the computation to deeper layers for all pieces and move types;
- The depth shift is especially pronounced for specific tactics, notably knight forks;
- The migration consists of deeper heads being recruited for more specialized computations, while a shared shallow head maintains a roughly constant contribution.
These results suggest that conditioning inputs can redistribute computation within larger Transformers, affecting both the depth and specialization of the model’s reasoning.
Review