Distilling long‑chain reasoning trajectories from large teacher models is a proven way to boost the reasoning ability of small language models. Existing pipelines usually fix the training set before distillation, ignoring that the usefulness of each example changes as the student model improves. This static selection creates two inefficiencies: examples the student has already mastered provide little new supervision, while examples far beyond the student’s current capability are hard to learn from, especially when only a limited number of teacher trajectories are available.
To tackle this, we introduce BoundaryEvo, a two‑stage data‑model co‑evolution framework.
Stage 1: Capability Priming directly distills a small collection of teacher‑generated long reasoning trajectories without any separate filter model, allowing the student to acquire an initial reasoning capability.
Stage 2: Capability‑Boundary Co‑Evolution repeatedly selects questions that the current student has not yet mastered but the expert can solve reliably, adds them to the training pool, and continues distillation. As the student improves, the data selection adapts to the evolving capability boundary.
With only 1,000 training examples, BoundaryEvo consistently outperforms both static and dynamic data‑selection baselines across 11 reasoning benchmarks. For the Qwen3.5‑4B model, accuracy on IMO‑AnswerBench rises from 62.9% to 75.3% and on AMO‑Bench from 26.8% to 46.0%. The code and data will be released publicly.
Review