A central challenge of on‑policy distillation (OPD) is extracting effective supervision from a teacher model. Using token probabilities alone often under‑utilizes the teacher’s internal knowledge, while directly matching hidden states forces the student into the teacher’s feature space and complicates cross‑architecture transfer.\ \ Inspired by contrastive learning’s exploitation of inter‑instance relationships, SimOPD transfers teacher knowledge via response similarities. It maintains paired memory banks of student‑generated responses and derives supervision from the teacher’s internal representations as similarity distributions. The student then aligns these distributions within its own representation space, preserving representational freedom and eliminating the need for cross‑model projection.\ \ Extensive experiments show that SimOPD improves the overall average performance by up to 2.6 percentage points in same‑architecture settings and 3.1 percentage points in cross‑architecture settings compared to the strongest baselines.\ \ Review