NeFut Logo NeFut
中 Admin Login

[CS.AI] SpatialASM: Internalizing Active Mentalization for Spatial Reasoning

Published at: 2026-10-14 22:00 Last updated: 2026-10-18 08:05
#AI #Machine Learning #LLM

When tackling complex spatial reasoning problems, humans naturally revisit scenes from task‑relevant viewpoints and mentally reconstruct the layout from those perspectives. We call this ability Active Spatial Mentalization (ASM), a core component of visual intelligence that is largely absent in current Large Vision‑Language Models (LVLMs). To fill this gap, we introduce SpatialASM, a scalable framework that internalizes ASM into LVLMs via training‑time spatial simulation.\ \ SpatialASM employs an allocentric novel‑view generator to anchor hypothetical viewpoints in existing visual evidence and produces two complementary supervision signals: (1) explicit ASM trajectories that teach LVLMs when and how to adaptively trigger view planning and imagination during spatial reasoning; (2) automatically constructed perspective‑taking tasks that further strengthen viewpoint‑conditioned scene imagination.\ \ We evaluate the approach on seven benchmarks ranging from perception‑oriented spatial understanding to more demanding tasks that require multi‑image, cross‑view, and viewpoint‑transformation reasoning. Using four different LVLM backbones, SpatialASM consistently improves accuracy across all benchmarks while preserving the models' original general vision capabilities.\ \ Moreover, the synthesis of ASM trajectories offers a scalable way to alleviate the scarcity of spatial Chain‑of‑Thought supervision, enabling models to actively explore viewpoints and simulate mental scenes.\ \ Review

Original Source: https://arxiv.org/abs/2610.13832

[h] Back to Home