Multi‑teacher on‑policy distillation (MOPD) merges several domain‑specialist policies into a shared student, improving cross‑task performance. The remaining challenge is how to weight each teacher’s supervision. Existing discrepancy‑based allocation may still apply strong supervision even when the student already matches or exceeds the teacher’s task performance, because behavioral mismatch does not necessarily indicate a capability deficit.
We introduce Capability‑Aware Multi‑Teacher On‑Policy Distillation (CA‑MOPD), which couples specialist merging with capability‑guided allocation to account for what the student has already learned. CA‑MOPD first merges reinforcement‑learning specialists into a strong student initialization, then uses a verifier to estimate teacher‑student capability gaps and adjusts domain‑level distillation strength accordingly. Supervision weakens as gaps close and strengthens again if the student’s capability regresses, while dense token‑level guidance from teachers is retained.
Across six benchmarks covering mathematics, code generation, and instruction following, CA‑MOPD achieves an average score of 32.22, outperforming naive MOPD, Open‑MOPD, and the merged‑initialization baseline by 0.86, 0.83, and 0.74 points respectively. Controlled comparisons further validate the benefits of merged initialization and capability‑gap guidance under equal training budgets.
Review