Text‑based world models have recently shown promise in simulating complex reinforcement‑learning environment state transitions, yet no standardized benchmark exists for systematic comparison and improvement. This paper introduces GlobeBench, a novel benchmark comprising 1,632 tasks across five domains and nineteen simulation skills. Each task asks a model to predict the environment’s response to a specific action given the full agent trajectory, requiring understanding of prior state changes, adherence to environment rules, and prediction of effects on both related and unrelated objects. Evaluation uses a hybrid architecture that combines deterministic gates with a semantic judge, comparing the model’s predicted response against an execution‑derived reference. Among 20 general‑purpose LLMs and specialized text world models, GPT‑6 Astra achieves the highest success rate at 49.57%. Accuracy declines as actions affect larger portions of the environment; when changes must propagate to dependent objects, the average success drops to 5.1% across all models. Although strengths vary per task, 631 tasks remain unsolved by any model, highlighting a substantial gap in general LLM capabilities as world models.
Review