Large language model (LLM) agents can handle increasingly complex tasks, yet effective assistance requires aligning their actions with user goals and constraints that may remain unstated. We treat this as interactive intent alignment via clarification. Clarification is useful only when the agent can turn the answer into better decisions. As training changes how the agent deals with incomplete information and leverages additional answers, the same clarification can become more or less valuable. To address this, we introduce CLEAT (CLarification through valuE‑guided Agent co‑Training), a framework that co‑trains a clarification‑value controller and the task policy from shared interaction outcomes. We define residual query value to measure the utility gain of asking a question versus proceeding under the current policy, and estimate it through paired rollouts generated from the same interaction history. These paired returns train the controller to predict clarification value while the task policy learns to select useful questions and act on the answers. The controller guides subsequent rollout collection, and outcomes from the updated agent provide new value labels, allowing clarification guidance to adapt as the agent learns. Using Qwen3‑4B, CLEAT achieves the highest mean score across all ten evaluation settings in four benchmarks. Gains over the strongest baseline reach 13.2 % and 6.5 % on UserGym Travel and IN3 underspecification judgment accuracy, respectively. Ablations show that co‑training improves the task policy even when the controller is removed at inference, and that adapting clarification guidance to the evolving policy yields further improvements.
Review