A simulator compresses an environment into a readable and mutable state. Generative models learn world representations without explicit specifications, yet they lack a built‑in way to edit that state. This work studies editability: whether changing a model’s latent representation yields the same next‑step outputs as editing a simulator’s state. The goal is to find a mapping between a model’s latent state and the simulator state, training the model only on observations and learning the mapping afterward. Two mappings are learned from observation‑state pairs: (1) train probes to decode the simulator state from the model and steer along it; (2) learn a direct inverse map, which can succeed even when steering fails. Experiments use two contrasting environments: the discrete board game Othello and Rayworld, a continuous world of moving discs observed through partial renderings, closer to physical scenes and video. Findings show that which mapping works depends heavily on the environment, although the state is decodable in every variant. In Othello, editability degrades smoothly as the game is simplified, and tiny rule changes have large effects. In Rayworld, probes cannot steer along the continuous state, yet an inverse map can still set the model from that state. Thus a generative model can be driven by a simulator it may never have learned.
Review