In inter‑example relational distillation the teacher’s representation geometry is transferred by matching relations among examples within a mini‑batch. Computing all pairs incurs quadratic cost in batch size, and uniform subsampling, while cheaper, often wastes a limited relation budget. We therefore propose Reliability‑Aware Pair Importance Distillation (RAPID), which separates a full‑support adaptive pair proposal from a reliability‑gated relational target. Reliability decides which teacher relations are emphasized; calibrated teacher entropy and detached student‑teacher residuals decide which relations are actually evaluated. An exact inverse‑proposal correction makes the loss and gradient estimators conditionally unbiased with respect to the gated mini‑batch target. We evaluate RAPID on two text‑classification setups: AG News with BERT→DistilBERT distillation (relation budget 256) and SST‑2 with DistilBERT→DistilBERT distillation (budget 64), each using three paired seeds. The reliability‑gated distillation achieves the highest mean student accuracy on both datasets, 94.285 ± 0.054 % and 88.800 ± 0.532 % respectively. In overall ranking RAPID is second, with 94.241 ± 0.025 % and 88.685 ± 0.462 %, compared to the cross‑entropy baseline’s 94.154 ± 0.124 % and 87.271 ± 0.162 %. Pilot evaluations count toward the same total budget as the main relation evaluations. Across settings the gated target consistently yields the highest mean accuracy, while the adaptive proposal stays within seed‑level variation. These results support the modular view that target reliability and evaluation priority are separable design dimensions.
Review: RAPID’s separation of target reliability and evaluation priority proves effective, delivering notable accuracy gains in distillation.