Abstract
Robotic manipulation policies rely on pre-trained vision models that provide either a global scene embedding or a dense patch grid, both mixing task-relevant and task-irrelevant features. Object-centric slot representations offer a structured alternative by grouping features into a few per-object slots. We test the value of this structure on ManiSkill3 PickCube-v1, using a frozen encoder and held-out-seed evaluation.
By holding the policy, goal token, rendering, and calibration fixed and changing only the encoder, a frozen object-centric SPOT representation (DINO ViT-B/16 + Slot Attention) achieves a success rate of 55.0$\pm$2.9%, which is 22.4% above a dense DINO global-feature baseline (32.6$\pm$1.5%), without encoder fine-tuning on the same trainable policy.
More tokens alone do not help: a dense patch grid with 16 times the tokens performs no better than the global feature. Adding an explicit 2D spatial goal and native-resolution rendering raises the full system to 68.7$\pm$4.2%, just below a privileged 3D-oracle upper bound (71.7$\pm$4.1%).
An automated kinematic failure taxonomy then separates spatial-precision (Near-Miss) failures from object-tracking (No-Grasp) failures: spatial grounding reduces Near-Miss while leaving No-Grasp unchanged. The same taxonomy transfers to the harder StackCube-v1 and points to occlusion as the main bottleneck.
Blogger's Review: This study demonstrates the advantages of object-centric representations in robotic manipulation, highlighting that leveraging structured feature groups significantly enhances performance. It suggests that effective use of structured information is far more crucial than merely increasing capacity in visuomotor imitation learning, potentially guiding future innovations in robotic learning strategies.