Large language models need to keep learning from interaction experience during deployment to improve continuously. Context distillation offers a viable approach: privileged information extracted from experience is fed to a teacher model, whose behavior is then distilled into a student model that cannot access the information directly. Existing off‑policy distillation trains the student on teacher‑generated trajectories, which introduces exposure bias, while on‑policy distillation trains on the student’s own rollouts, whose states are often uninformative and fail to convey behaviors the student has not yet mastered. To address this dilemma, we propose Localized Context Distillation (LoCD). LoCD first identifies the critical spans in a student trajectory that need revision. The teacher then regenerates only these spans from the student’s prefix, and the student is trained on the regenerated segments. This concentrates learning on the most informative states while keeping exposure bias low. Extensive experiments on math reasoning, code generation, and agent tasks show that LoCD consistently outperforms strong baselines and achieves more stable training with smaller and sparser parameter updates.
Review