Abstract
Proprietary large language models (LLMs) entail substantial intellectual and financial investment, making them valuable intellectual property (IP). However, even when deployed via black-box APIs, these models remain vulnerable to unauthorized knowledge distillation, allowing adversaries to cheaply extract and replicate model capabilities. To address this, anti-distillation (AD) has been proposed to generate defensive outputs that hinder distillation effectiveness, overcoming the limitations of watermarking-based approaches that rely on post-hoc verification.
Existing AD methods based on internal model perturbations struggle to balance anti-distillability and utility (e.g., answer accuracy and naturalness) of reasoning traces, with stronger defenses often causing significant utility loss. To fill this gap, we propose Skeleton-Guided Reasoning Editing (SGRE), an Answer-then-Edit framework that performs post-hoc trace modification for anti-distillation.
In the answer stage, the teacher model first generates clean reasoning traces, preserving the original reasoning accuracy while enabling more flexible control over trace naturalness. In the editing stage, we draw inspiration from Cognitive Load Theory (CLT) and introduce a three-stage strategy consisting of reasoning skeleton extraction, skeleton graph coarsening, and skeleton verbalization. These operations jointly perturb reasoning structures and augment textual complexity to amplify extraneous load on student models, hindering their acquisition of underlying reasoning patterns.
Extensive experiments across diverse LLMs demonstrate that SGRE achieves state-of-the-art performance in reducing distillation effectiveness, while maintaining lossless reasoning accuracy and superior trace naturalness.
Blogger's Review: The SGRE framework offers an effective solution in the anti-distillation domain through innovative reasoning skeleton editing methods, maintaining model utility while enhancing defensive capabilities. This research not only strengthens the protection of LLMs but also paves the way for new approaches to intellectual property protection in the future.