Background
Egocentric human data offers scalable supervision for robot manipulation. However, behavior cloning entangles transferable content like objects, scenes, and task semantics with non-transferable factors such as human morphology, head motion, and behavioral style. We investigate whether World Action Models (WAMs) provide a better training signal by requiring policies to predict not only actions but also how the scene evolves.
Research Objective
The central question is what world representation best enables human-to-robot transfer. We hypothesize that an effective world target should abstract appearance, capture agent-invariant physical effects, and separate camera motion from environment change.
EgoWAM Framework
We introduce EgoWAM, a controlled human-robot co-training framework that fixes the policy backbone, action head, and data mixture while varying only the world prediction target, comparing Pixel, DINO, and 3D motion flow. Across three real-world bimanual tasks, WAM co-training scales more effectively with in-the-wild egocentric human data than behavior cloning.
Pixel-based prediction transfers weakly, while DINO and 3D flow yield substantial gains: DINO improves out-of-distribution object and scene generalization by up to 4x, and 3D flow improves in-domain performance by 20-30%.
Conclusion
The EgoWAM framework demonstrates how selecting appropriate world representations can optimize robot learning in complex environments.
Blogger's Review: The study of EgoWAM highlights that choosing the right world model can significantly enhance performance in robotic learning, especially in complex real-world contexts. This provides a fresh perspective for the future development of robotic intelligence, particularly in utilizing egocentric data effectively.