In this work the authors study on‑policy distillation (OPD) for reasoning models. Standard OPD improves a student by learning the token‑level discrepancy between a stronger teacher and the on‑policy student, but this discrepancy mixes the teacher’s own self‑deviation, so the student inadvertently learns the teacher’s errors. Privileged OPD aggravates the issue because privileged information causes larger likelihood shifts on the teacher side. To address this, the authors propose Calibrated On‑Policy Distillation (Cal‑OPD). By applying positive and negative privileged interventions they estimate the teacher’s self‑deviation region and keep only the component of the teacher‑student discrepancy that lies outside this region as the training signal. Experiments on mathematical reasoning benchmarks show that, while retaining only about 52‑65% of the original discrepancy, Cal‑OPD consistently outperforms vanilla OPD and its variants across model scales.
Review