Behavioral failures can make a transformer appear to lack a world model even when it has learned faithful representations of its environment. We demonstrate this phenomenon with TaxiGPT, a transformer trained on random walks through Manhattan whose failures were previously taken as evidence of an incoherent internal map. By means of mechanistic analysis and causal interventions we show that the model encodes intersections and streets, tracks its own position, and employs a goal compass for navigation. The failures stem from interference among superposed intersection features, which disrupts localization within the internal map.
To mitigate the impact of such errors the model uses an "affordance packing" strategy: representations of intersections that share the same legal moves are grouped together, limiting error propagation. We also propose a set of mechanistic indicators to compare models and reveal that world‑modeling capacities emerge at different stages of training.
These findings motivate a shift from asking whether a model has a world model to mechanistically studying its world modeling: the interacting capacities through which it represents its environment and uses those representations to guide behavior.
Review