Continual post‑training must acquire new capabilities while retaining prior knowledge. Self‑Distillation Fine‑Tuning (SDFT) uses in‑context demonstrations for self‑supervision and works well on large models, but it relies on strong in‑context learning and instruction‑following, causing failure on the smaller models needed for low‑latency, on‑device scenarios.
We first pinpoint the failure cause: both poor acquisition and forgetting correlate linearly with the distribution shift of the post‑trained model, indicating that learning must stay in‑distribution. To address this, we introduce Synthetic Self‑Distillation (SynthSelf). A minimally larger, same‑family aligner generates high‑quality, in‑distribution rewrites, which are then used to train the small model. Off‑policy supervision and rejection sampling are added to preserve the original distribution while providing reliable supervision.
Evaluations on four acquisition datasets (including a mixture) and fourteen retention benchmarks show SynthSelf consistently outperforms six baselines (both off‑ and on‑policy distillation) across three small‑model scales. Ablation studies confirm that in‑distribution rewrites, off‑policy supervision, and rejection sampling are the main contributors to the gains.
In summary, SynthSelf opens a practical route for effective continual learning in small models, overcoming the limitations of traditional self‑distillation.
Review