On-Policy Distillation (OPD) has become a widely adopted post‑training paradigm for improving large language models in reasoning tasks, yet its data‑centric mechanisms are still under‑explored. This work starts with an extreme setting: training OPD on a single example, i.e., 1‑shot OPD. Surprisingly, 1‑shot OPD consistently boosts performance across all sampled training examples, and harder examples tend to yield larger gains.
To understand what drives the student’s improvement, we analyze the training data and find that high token entropy is not the key factor. Instead, longer chain‑of‑thought (CoT) paths—naturally produced by difficult problems—play a decisive role. Training on longer CoTs helps the student stay aligned with the teacher over a longer reasoning horizon and acquire critical thinking patterns often missing in short CoTs, such as reflective cues like “Alternatively”.
Guided by these insights, we propose a simple data‑selection method that retains only hard examples for training, even allowing “unsolvable” cases that exceed the teacher’s capability. Experiments on four models ranging from 1.5B to 7B parameters show that training on just eight selected hard examples matches the performance of the 17K‑example baseline.
Review: The paper reveals that the length of CoT, rather than token entropy, is the primary driver of OPD effectiveness, and demonstrates that a tiny set of hard examples can achieve data‑efficient distillation, offering a clear and practical direction for future large‑model distillation research.