In the field of robotics, World Models (WMs) are increasingly employed to evaluate action policies by simulating the consequences of actions in an imagined world and returning a success or safety verdict. However, the trustworthiness of a verdict is only as reliable as the WM that produced it, and the WM itself needs to be certified. In video-generating WMs, fidelity metrics like Fréchet Video Distance (FVD) reward visual realism but neglect whether the world responds correctly to the policy's actions, including those unseen during training.
Classical simulation-based validation assumes a trusted simulator evaluating an untrusted policy, while generative WMs are themselves unverified learned artifacts. Therefore, we argue that any WM used as a test oracle must first be accredited before its verdicts can serve as evidence. Drawing from credibility practices in safety-critical simulation, including Verification, Validation & Accreditation (VV&A), Safety of the Intended Functionality (SOTIF), and scenario-based testing standards, we define an admissibility ladder (L0-L4) that a WM must climb before its closed-loop verdicts are accepted as assurance evidence.
Our framework is embodiment-agnostic and is instantiated in autonomous driving (AD), where assurance methods for traditional simulation are most mature. When applied to two driving WMs, the lower rungs reveal a reversal: the model that ranks higher on visual generation quality (L0) ranks lower on action-following (L1-L2), indicating that visual fidelity does not predict the action robustness a closed-loop verdict depends on.
Blogger's Review: This study underscores the necessity of validating models when using World Models for decision-making, particularly in safety-critical applications. Relying solely on visual quality may lead to unreliable outcomes, providing crucial guidance for future model development and evaluation. By introducing an admissibility ladder, the research offers a clear framework for systematically validating and certifying WMs, which is worthy of broader application and discussion.