NeFut Logo NeFut
Admin Login

[CS.AI] Environment Evolution for Terminal Agents

Published at: 2026-09-04 22:00 Last updated: 2026-09-05 12:23
#AI #Machine Learning #LLM

Scaling interactive and verifiable environments is essential for training terminal agents. As frontier models become more capable, environments built from scratch lose difficulty and provide weak learning signals. Existing co‑evolution methods iteratively synthesize environments near the model’s learnable frontier by exposing weaknesses during on‑policy rollouts, but this reliance limits generalization and continuous signal provision as the model strengthens.\ \ We introduce environment evolution, which incrementally raises environment difficulty off‑policy and schedules the evolved environments generation‑by‑generation during training, ensuring a steady stream of learning signals. From the multi‑turn learning objective we derive three evolution directions that affect difficulty, and we implement evolution along these axes using a loop‑engineered multi‑agent harness.\ \ Quantitative rollout experiments with Hy4 preview, Claude Opus 5, and GPT‑5.6 Sol demonstrate that environment evolution consistently yields harder environments. Validation on Qwen3.6‑27B and Qwen3.6‑35B‑A3B through simple long‑horizon RL training improves their scores on Terminal‑Bench 2.1 by 14.4 and 18.0 percentage points, respectively.\ \ Review

Original Source: https://arxiv.org/abs/2609.04128

[h] Back to Home