On‑policy distillation (OPD) can achieve strong results with surprisingly few training questions, suggesting that questions act as carriers of distributional knowledge transfer. To assess the value of a newly generated question for distillation and to learn to produce better questions without access to correct answers, we introduce Ask‑to‑Distill (\method). The framework treats synthetic questions as distribution elicitors and optimizes the teacher supervision they expose along student‑generated trajectories.
In \method, a student‑initialized synthesizer is optimized using a distributional surrogate value computed from a frozen student‑teacher pair, requiring no reference answers, correctness rewards, or per‑candidate student updates. The reward combines three components:
$$R = \lambda_1 \cdot \text{Disagree}(T, S) + \lambda_2 \cdot \sum_{a \in \mathcal{C}} P_T(a) - \lambda_3 \cdot \text{len}(q) + \lambda_4 \cdot \text{Div}(\mathcal{C})$$
Here $\text{Disagree}(T, S)$ measures teacher‑student disagreement over the candidate answer set $\mathcal{C}$, $\sum_{a \in \mathcal{C}} P_T(a)$ captures the teacher’s probability mass on the student’s support, $\text{len}(q)$ penalizes question length, and $\text{Div}(\mathcal{C})$ encourages answer diversity.
Generated questions are filtered and deduplicated into a prompt‑only bank for standard OPD training. Across mathematical reasoning, code generation, and agentic tasks, \method improves average performance and reduces the teacher‑student gap compared to unoptimized student‑generated prompts and KL‑filtered real questions, matching or exceeding real‑bank OPD in most settings. Signal analysis shows that optimized questions elicit denser supervision from shorter student responses, supporting distributional supervision as an objective for synthetic distillation data.
The code, data, and models are publicly available at https://github.com/GasolSun36/Ask_to_Distill.
Review: \method reframes questions as information carriers, enabling automatic evaluation and improvement of synthetic data without answer labels, and offers a promising direction for efficient on‑policy distillation.