Abstract
High-volume structured extraction incurs the latency of a large model on every item, making the distillation of the task into a small on-device model appealing: comparable output at a fraction of the time and cost. We measure what this distillation actually delivers per sub-task. Each news article is mapped to one JSON object with a short summary and five categorical labels.
We distill an 8B reasoning teacher (deepseek-r1:8b) into a 0.6B student (Qwen3-0.6B; QLoRA, three seeds), adding two teacher controls: a same-size non-reasoning teacher and a larger managed pipeline. A blinded, reference-free, three-judge panel scores each arm against the full article, alongside two non-distillation baselines: few-shot prompting and constrained decoding.
The student runs at about 0.8 seconds per article compared to the teacher's 39 seconds, recovering 58% of the base-to-teacher gap on summary quality, beating its primary baseline (constrained decoding) by +16.8 points and few-shot prompting by a secondary +4.9 points. A same-size non-reasoning teacher trains a student no better than the untuned base, indicating that the summary gain comes from the teacher's reasoning nature rather than its scale. Capabilities split by teacher: the reasoning teacher transfers writing quality while the managed pipeline transfers label diversity, whereas a same-size instruction teacher's students remain more grounded on the 22 short, thin-source articles in the 93-item test set (74 versus 55 faithful), where the reasoning-lineage student fabricates.
This grounding difference is a consistent ordering rather than a significant aggregate effect and the subgroup is small, so we report it as a direction. As no single engine excels in every field, the deliverable is a per-field routing map for on-device enrichment.
Blogger's Review: This study reveals the potential and limitations of models in structured text extraction through the comparison of different types of distillation teachers. The advantages of reasoning teachers highlight the necessity of intelligent models in complex task handling, providing significant insights for future text processing and generation technologies.