NeFut Logo NeFut
Admin Login

[CS.AI] State-Matched Routing and Contextualized Self-Distillation for Multi-Turn Agents

Published at: 2026-08-07 22:00 Last updated: 2026-08-08 01:08
#Machine Learning #Artificial Intelligence #DeepSeek

Privileged on-policy distillation provides dense supervision for multi-turn agents by allowing a synchronized teacher to re-score the student's response at every turn with access to training-only references, such as successful trajectories. However, the student's preceding actions continually change the execution state, making the reference an unreliable source of guidance for the state actually reached. To address this issue, we introduce State-Matched Routing and Contextualized Self-Distillation (SMRC-SD), which explicitly determines when and how a privileged trajectory should guide an on-policy student. At each turn, SMRC-SD verifies whether the student's current execution state matches a supported state along the reference trajectory. Distillation is applied only at matched states, filtering out turns for which the reference lacks locally compatible guidance. For each matched state, SMRC-SD further constructs state-conditioned teacher context from the successful trajectory, grounding supervision in the state actually reached. Across ALFWorld and WebShop, SMRC-SD consistently outperforms unconditional successful full-path distillation. With Qwen3-1.7B, it improves task success from $0.746$ to $0.865$ on ALFWorld and from $0.574$ to $0.693$ on WebShop. Controlled routing and context ablations support both selecting locally supported turns and constructing state-compatible teacher context as contributors to these gains. The code is available at https://github.com/liujunzhuo/SMRC-SD. Blogger's Review: This paper proposes a novel approach to address the mismatch issue of privileged guidance in multi-turn agents, and achieves significant performance improvements through state-matched routing and contextualized self-distillation,

Original Source: https://arxiv.org/abs/2608.05219

[h] Back to Home