The performance of an LLM agent largely depends on the scaffold built around a frozen model. A typical way to improve this scaffold is to employ a coding agent as an optimizer: it reads current scores and execution traces, iteratively edits the source code, and produces a new candidate each round. Each edit is chosen based on a belief about how the environment will react—what went wrong and which change should help. This belief is usually implicit, residing in the coding agent's on‑the‑fly reasoning or its parameters, rather than being recorded explicitly. Consequently, later calls only see scores and traces but cannot directly use the prior belief.
We introduce Belief‑Calibrated Optimization (BCO), which writes that belief into a persistent in‑context document and continuously revises the document as new candidates are evaluated. The document serves as a world model, capturing the current account of how the environment responds to edits. Adding BCO to a standard optimization loop yields a higher training pass rate than a matched control lacking only the world model, across five benchmarks covering memory QA, tool‑use QA, code‑as‑action app agents, and terminal agents. The gap persists on every held‑out split, which is not used during candidate selection.
In a target‑model swap experiment, where the frozen model is replaced while the scaffold remains unchanged, the BCO‑generated scaffold still leads on the tested tasks, except where context‑window overruns cause incomplete execution.
An offline ablation then asks whether the performance gain stems from the world model’s content. A fresh predictor given the accumulated document forecasts the environment’s response more accurately than predictors given either no document or a same‑form copy whose content has been falsified. This comparison indicates that the document carries reusable information in its content, not merely in its form.
Review