Current world models typically operate at a single level of abstraction, prioritizing perceptual fidelity while lacking the spatial reasoning and semantic understanding necessary for real-world downstream tasks. We present a hierarchical driving world model that factorizes future prediction across two levels operating at distinct temporal and abstraction scales: a high-level predictor that forecasts coarse scene structure over extended temporal horizons, and a low-level generator that produces detailed predictions conditioned on the high-level output. This decomposition yields high perceptual fidelity while also capturing strong spatial and semantic representations.
We further demonstrate that pretraining with a diffusion forcing objective yields substantially richer internal representations than the standard teacher forcing objective, while teacher forcing—predicting only the next frame from clean context—produces more stable autoregressive rollouts. Therefore, we introduce a generic two-stage training paradigm that pretrains the model with diffusion forcing and fine-tunes with teacher forcing, combining the representational benefits of the former with the rollout stability of the latter.
Our approach achieves state-of-the-art results across the standard suite of driving world model evaluations on established benchmarks, including long-horizon generation fidelity, steering responsiveness evaluated on counterfactual scenarios, and internal representation quality. Project page with code, demo, checkpoints, and qualitative results: Orbis 2 Project
Blogger's Review: Orbis 2 enhances the performance of driving world models through a hierarchical design, particularly excelling in long-term prediction and spatial understanding, showcasing unprecedented potential. This innovation not only advances the development of autonomous driving technology but also offers new insights for related research fields. Its effective two-stage training approach could become a mainstream trend in future model training.