NeFut Logo NeFut
中 Admin Login

[CS.AI] Interactive World Simulation with Cosmos for Physical AI

Published at: 2026-11-06 22:00 Last updated: 2026-11-07 12:32
#AI #Machine Learning #Open Source

We introduce a shared framework that adapts Cosmos world foundation models for real‑time interactive simulation in Physical AI. By streaming visual observations in response to actions and scene controls, the framework lets users and embodied agents interact with continuously evolving virtual environments. It supports both predicting the visual consequences of actions and generating observations from physical states maintained by an external simulator. We demonstrate the approach on camera navigation, robot manipulation, controllable video transfer, and multi‑view autonomous‑vehicle simulation. These examples offer a common foundation for building interactive virtual environments for Physical AI training and evaluation. The code, model checkpoints, and project website are publicly released for community use.

Review: By bridging visual streams with external physics simulators, this framework lowers the barrier to creating interactive simulation environments, providing a reusable infrastructure for large‑scale Physical AI research.

Original Source: https://arxiv.org/abs/2611.06627

[h] Back to Home