Despite the remarkable progress of multimodal large language models (MLLMs), understanding 3D spatial relationships from 2D images remains a significant challenge. Existing methods primarily rely on symbolic text tokens, which inherently lack the fidelity to represent continuous geometric information. Although recent approaches leverage latent representations to enhance reasoning, relying on a single latent type fails to adapt to the diversity of spatial tasks, causing misalignment in complex geometric scenarios.
To tackle these limitations, we propose GeoAnchor, an interleaved text-latent reasoning framework. GeoAnchor decomposes 3D spatial information into three complementary components: position latents for object grounding, direction latents for relational orientation, and geometry latents for scene structure. These components are recombined in a structured space to construct local evidence while capturing global context, enabling dynamic and interpretable reasoning.
Additionally, we introduce a collaborative training strategy that guides the model from local spatial perception to comprehensive 3D understanding. Extensive experiments on diverse and complex 3D reasoning tasks demonstrate that GeoAnchor outperforms the state of the art, validating its effectiveness and generalization capabilities.
Blogger's Review: GeoAnchor significantly enhances the ability to understand 3D spatial structures through latent decomposition, showcasing the vast potential of multimodal reasoning. Its innovative collaborative training strategy points the way forward for future research, making it a noteworthy contribution.