World models enable agents to compare imagined outcomes before acting, yet the agent must still choose which alternatives to explore and when a proposed plan should replace its learned policy. We introduce SDI (Sibling Disagreement in Imagination), which links these decisions through a shared actor‑based reference. SDI compares action sequences imagined from the same model state, using repeated rollouts of the actor’s own sequence to measure variation under fixed behavior. This reference both amplifies the spread of candidate returns—providing an exploration signal—and quantifies a plan’s predicted advantage, governing action acceptance. The agent executes a planned action only when its advantage exceeds the reference variability by a fixed factor; otherwise it follows the actor. SDI reuses the base agent’s world model, actor, and critic without requiring an additional disagreement ensemble. Exploration and planning jointly guide training data collection, while evaluation relies solely on the learned actor. Across Meta‑World, Adroit, and DM‑Control, SDI attains the highest observed task‑average score among all baselines in each suite and matches or exceeds the strongest baseline on 15 of 18 tasks. Component ablations and robustness analyses further clarify the roles of exploration, planning exposure, and action acceptance. Paired evaluations from identical simulator states show that the gating improves both recovery from actor failures and preservation of successful behavior relative to ungated planning, substantially reducing return degradation. Together, these findings support comparing imagined alternatives against a shared actor reference as a unified approach to exploration and planning with world models.
Review