NeFut Logo NeFut
中 Admin Login

[CS.AI] GRADUS: Discovering and Recovering Agent Lessons from the Environment's Latent Syllabus

Published at: 2026-10-22 22:00 Last updated: 2026-10-25 13:04
#algorithm #Machine Learning #LLM

Training large language model (LLM) agents for policy‑constrained tool environments—such as an airline backend with booking, refund, and cancellation rules—requires tasks that exercise each rule individually and in combination. Existing synthesis pipelines let a model generate tasks and keep only those it solves immediately, biasing the training set toward easy behaviors.\ \ Gradus (Graded Rungs of Ascending Difficulty from an Underlying System) separates what a task should teach from the concrete task and solution used to teach it. Without any seed tasks, Gradus treats a fixed environment as a latent syllabus, extracting guarded behavioral operators that capture how actions depend on current conditions and how their effects reshape later decisions. These operators are then composed into progressively harder tasks, while corpus‑level feedback targets missing outcome combinations. Crucially, Gradus jointly calibrates tasks and solutions against the environment: execution feedback indicates whether to refine the task or the solution, preserving the intended behavioral requirements. Verification thus becomes a recovery signal rather than a terminal filter, coupling coverage‑guided exploration with the preservation of difficult examples.\ \ Using off‑the‑shelf backbones, Gradus lifts a 30B model from 59% to 89% on $\tau^2$‑Bench and from 47% to 83% on CAR‑Bench with an Opus 4.5 proposer, and to 76% and 62% respectively with an open‑weight Qwen3‑235B proposer. These results demonstrate the value of jointly controlling which behaviors are explored and which survive into verified training data.\ \ Review

Original Source: https://arxiv.org/abs/2610.27781

[h] Back to Home