In reinforcement learning, the interaction horizon denotes the maximum number of steps an agent can interact with the environment within a single episode. Recent work shows that enlarging this horizon markedly improves large language model (LLM) agents on long‑horizon tasks. Conventional approaches use a fixed horizon or a preset monotonic schedule, which are open‑loop: the horizon grows until a manually chosen maximum, without a mechanism to detect when further expansion ceases to help.
We propose the effective interaction frontier hypothesis: there exists a dynamic boundary beyond which each additional interaction yields diminishing returns while computational cost grows linearly. Guided by this hypothesis, we introduce Elastic Horizon, a closed‑loop controller that tracks the 90th percentile of successful trajectory lengths to estimate the frontier in real time.
On the AppWorld and BFCL benchmarks, we first conduct exhaustive sweeps over fixed horizons and observe clear saturation plateaus in success rates. Starting Elastic Horizon from both under‑ and over‑capacity initial horizons, the controller stabilizes the horizon within the saturation band, avoiding wasteful computation and achieving higher success. Compared with fixed horizons, Elastic Horizon attains the best success rates on both 7B and 14B backbones and reduces per‑step trajectory token consumption by up to 25%.
Our work shifts the focus from how to scale interaction horizons to when to stop scaling, offering a new perspective for efficient training of large‑scale LLM agents.
Review