Spatial reasoning is essential for vision‑language models (VLMs) to understand and act in the physical world. Dynamic environments require models to perceive local state changes caused by object motion and viewpoint shifts, and to maintain an updated spatial state over long trajectories. Existing VLMs, however, are limited in both aspects. Current spatial training mainly targets static questions about attributes and relations, offering little direct supervision for state transitions. In contrast, interaction trajectories naturally link a prior observation, an action, and a subsequent observation, providing direct supervision for local transitions, while full trajectories expose dependencies among consecutive changes.
We therefore introduce Spatial-Interactor, a framework that learns physical‑world state transitions through interaction. The learning process is organized into a three‑level curriculum: L1 passive world‑state transitions, L2 active self‑state transitions, and L3 long‑horizon interaction trajectories. Based on this, we build the Learning from Spatial Interaction dataset (LSI-108K) from simulated and real interaction trajectories, with tasks aligned to each level.
Training follows a two‑stage strategy. The first stage applies Supervised Fine‑Tuning (SFT) to L1 and L2 for local transition modeling. The second stage uses On‑Policy Distillation (OPD) with privileged self‑distillation: a teacher branch supplies segment‑level transition descriptions to supervise the student’s on‑policy chain‑of‑thought (CoT), helping the student integrate consecutive transitions over L3 long trajectories.
Experiments across multiple VLMs and spatial benchmarks show consistent gains in both local transition modeling and long‑horizon integration.
Review