On‑Policy Self‑distillation (OPSD) lets a large language model act as both teacher and student under different conditioning, enabling self‑improvement. However, we observe that as training proceeds some queries that were answered correctly early on become wrong, even though overall metrics keep rising. This degradation aligns with a collapse of hidden‑state geometry: intra‑cluster compactness drops and inter‑cluster separation shrinks. To counteract this, we introduce Representation Geometry Regularization (RGR), which adds a regularization term that forces representations of the same cluster to stay tight while pushing different clusters apart. RGR plugs into existing OPSD pipelines without altering their main optimization steps. Extensive experiments across several OPSD methods and benchmarks show that RGR consistently mitigates capability loss while preserving gains. The implementation is released at https://github.com/supermachinelearner/RGR.
Review