Self‑distillation can boost reasoning without a separately trained, more capable teacher, yet its success hinges on how the self‑teacher gains an advantage over the student. Conventional approaches supply this advantage via reference answers or solutions, which are often unavailable. Reflection can extract explicit error diagnoses and revision guidance from the model’s own attempts, but existing reflection‑based methods still rely on reference information, rich task feedback, or cross‑example memory.
ReTeach introduces a purely self‑generated, outcome‑level verification framework that builds a self‑teacher through multi‑round reflection and retry. Starting from a failed student rollout, the teacher alternates explicit reflection with renewed attempts until success or the retry budget is exhausted, without using reference answers, external diagnostics, or persistent memory. Each failed retry informs the next reflection, while a successful correction supplies outcome‑level evidence of the teacher context’s utility.
An outcome‑aware selection and weighting scheme categorizes examples into initially correct, reflection‑corrected, and unresolved groups, assigning separate normalized distillation loss weights to each. On‑policy distillation then aligns the student’s token‑level predictive distribution at prefixes of its own rollouts with the teacher’s context‑conditioned distribution, transferring the benefits of iterative correction while preserving single‑pass inference.
Across six benchmarks spanning mathematical reasoning, science QA, and tool use, ReTeach improves average accuracy over GRPO by 1.39 percentage points, demonstrating the effectiveness of multi‑round reflection‑retry in constructing a self‑teacher.
Review