NeFut Logo NeFut
Admin Login

[CS.AI] GeoProp: Aligning Robot State with Vision for Manipulation

Published at: 2026-07-10 22:00 Last updated: 2026-07-13 08:26
#AI #Machine Learning #optimization

Proprioception is fundamental to robotic manipulation, yet standard fusion methods often treat it as an isolated vector lacking explicit alignment with visual tokens.

Without a direct correspondence between 3D kinematics and 2D feature maps, manipulation policies struggle to ground the robot's state within the scene, frequently underperforming even vision-only baselines.

To address this, we introduce GeoProp, a lightweight, plug-and-play adapter that aligns proprioception with vision through explicit geometric grounding and spatial feature sampling. GeoProp projects the robot state onto the image plane to sample localized visual features, constructing a grounded state token.

It then injects state-derived spatial priors into the corresponding visual features via FiLM modulation. To capture motion intent, GeoProp further samples features at a short-horizon predicted coordinate derived from recent kinematics, providing look-ahead visual context.

Across 67 tasks, GeoProp improves Diffusion Policy by 8.7% on 63 simulation tasks and pi_0 by 4.0% on the RoboTwin subset, yielding a 10.6% average gain across both policy families in the real world, while adding only 2-3% to the parameter count.

These results demonstrate that GeoProp is a simple yet high-impact inductive bias for generalist embodied policies. Project page: GeoProp Project.

Blogger's Review: GeoProp effectively enhances the accuracy and efficiency of robotic manipulation by integrating proprioception with visual information. Its lightweight design offers strong flexibility for real-world applications, paving the way for innovative developments in robotics technology.

Original Source: https://arxiv.org/abs/2607.07101

[h] Back to Home