On‑policy distillation (OPD) improves a student language model by training on its own generated trajectories with supervision from a stronger teacher. However, when the teacher is conditioned on student‑generated prefixes, it may favor continuations that violate the question premises, leading to erroneous reasoning steps. To address this, we introduce Condition‑Contrastive Extrapolated On‑Policy Distillation (CCE‑OPD), which makes teacher supervision sensitive to question conditions. CCE‑OPD consists of two main components:
- Fixed‑prefix comparison: the same frozen teacher evaluates the full question and a counterpart with a selected condition masked, capturing the effect of the condition.
- Bounded log‑probability contrast: the teacher distribution is re‑weighted based on the comparison, forming an auxiliary distillation target.
On 200 diagnostic reasoning states, condition contrast reduces the rate of preferring invalid continuations from 27.0% to 16.0%. Extensive experiments across six mathematical benchmarks show that CCE‑OPD improves avg@8 over OPD by 1.60 percentage points when distilling Qwen3‑8B into a 0.6B student and by 1.95 points for a 1.7B student, and it achieves the strongest results on four transfer benchmarks.
Review