Web world models are increasingly used to forecast the outcomes of candidate actions before execution, yet a semantically plausible future does not guarantee a correct safety decision. In a matched human evaluation of 3,000 predictions from ten frozen open models, 86.1% were judged plausible. Several models, however, showed highly asymmetric authorisation behaviour: low false‑safe execution combined with poor recall on clarifiable ASK cases and a strong bias toward BLOCK. This motivates the notion of safety sufficiency, i.e., whether a predictive interface preserves and exposes the policy‑decisive information needed to distinguish actions that may be EXECUTED, require clarification via ASK, or should be BLOCKED.
We introduce the SafeWebWorld benchmark, covering five web‑safety domains and evaluating safety sufficiency through action‑conditioned futures, policy‑decisive evidence, and policy‑conditioned decisions. Under a controlled Security‑Aligned Supervised Fine‑Tuning (SASFT) protocol, evidence preservation improves markedly while authorisation‑sensitive decisions deteriorate. To determine whether this deterioration stems from loss of authorisation‑sensitive score structure or from a decision readout failure, we propose Latent Authorisation Recovery (LAR).
Frozen‑score recalibration reveals substantial but backbone‑dependent recovery, whereas paired recovery yields only modest additional gains. Nevertheless, within‑family and matched‑authorisation evaluations remain poor, demonstrating that aggregate recoverability does not imply robust fine‑grained authorisation behaviour. In sum, plausible prediction, evidence preservation, and aggregate recovery each fall short of establishing reliable authorisation.
Review: The paper’s benchmark and recovery analyses expose critical gaps in safety‑oriented prediction models, urging the community to balance high‑quality forecasting with thorough information retention and authorisation discrimination.