Existing large‑language‑model (LLM) based agents for long‑horizon machine learning engineering typically rely on context engineering, i.e., custom harnesses that orchestrate tools, skills, and accumulated experience. When a single session is prolonged, the agent’s reasoning intensity drops and progress slows, a phenomenon we call long‑horizon deceleration; as the underlying model improves, the advantage of the harness fades, termed harness staleness. These observations suggest that such benefits may not survive model upgrades, motivating a workflow‑level approach where agents build on each other’s work across sessions.
We model alternating sessions as a context‑reset semi‑Markov decision process (CR‑SMDP), derive sufficient conditions under which alternation outperforms a single‑agent reference, obtain an explicit experiment‑budget bound, and characterize how a session cap of at most $k$ experiments preserves the advantage. Guided by this analysis we introduce Humanize MLE Agents (HMA), which alternates two agents on a shared workspace, limits each session to at most $k$ experiments, and resets the context between sessions, without any task‑specific harness or prior knowledge supplements.
Across 75 Kaggle tasks in MLE‑bench, HMA achieves a state‑of‑the‑art any‑medal rate of 78.2%, exceeding the strongest open‑source baseline by 8.0 points while using only a quarter of its time budget. Across six configurations spanning five heterogeneous agent pairs, HMA improves the any‑medal rate over the constituent baselines by an average of 6.4 points, demonstrating cross‑pair generality. A supplementary kernel‑optimization case reaches 878 cycles, within 1.6% of the community leaderboard leader, showing applicability beyond MLE.
Review