Long‑horizon language agents increasingly rely on external memory as a frozen world model to handle complex environments. Existing memory systems are usually evaluated only by task success or token cost, overlooking the shape of memory usage. We argue that under finite context windows and repeated retrievals, an agent’s memory tends to concentrate on a small core of states while pushing rare states into a long tail, where prediction errors accumulate. To investigate this, we design a conservative tail audit that reveals a reproducible core‑tail distribution, yet strongly policy‑dependent. Random‑walk policies generate retrieval artifacts compatible with a log‑normal distribution, whereas semantic LLM policies exhibit the strongest truncated‑power‑law‑compatible core‑tail traces. Motivated by these findings, we propose the Core‑Tail World Model (CTWM), a rank‑based memory controller that allocates the prompt budget with a single exponent $\tau$ while retaining a summarized tail. On the Synthetic Graph World benchmark, CTWM preserves full state and transition coverage, reduces prompt tokens by 5.9% and lowers the bottom‑half tail prediction error by 13.6% relative to a graph‑memory baseline. The same paired comparison yields consistent token savings on ALFWorld and a 24.48% token reduction on LongMemEval while maintaining aggregate accuracy parity. These results suggest that heavy‑tailed memory traces are not only a diagnostic of finite retrieval but also a practical control signal for token‑efficient agent world models.
Review