State‑of‑the‑art language‑model agents excel at complex tasks, yet their inference cost is prohibitive, making it essential to transfer capabilities to smaller models. Conventional knowledge distillation does this by updating parameters, which requires access to the student’s weights, extra training compute, and encodes the transferred knowledge implicitly in the parameters.
D‑Skill treats the teacher as a black box. It analyzes paired rollouts to spot behavioral divergences and agreements, then converts the teacher’s behavioral advantage into an interpretable natural‑language skill description. The skill document is consolidated offline and can be fed to the student at inference time via any compatible skill‑update backend. The whole pipeline relies only on fixed trajectories—no labels, no further environment interaction, and no weight updates after rollout collection.
Across SpreadsheetBench‑Verified, ALFWorld, and chess‑puzzle benchmarks covering multiple teacher‑student pairs, D‑Skill closes 75%‑95% of the teacher‑student performance gap without touching model weights. After a single offline pass over the paired traces, D‑Skill outperforms one‑epoch supervised‑fine‑tuning (SFT) baselines and remains competitive with multi‑epoch SFT. Within the same model family, the resulting student incurs 5.7×‑8.3× lower inference cost per task than the teacher.
The results demonstrate that natural‑language skill distillation provides a performant, interpretable, and efficient alternative to weight‑based capability transfer.
Review