This paper proposes an improved structured pruning method that addresses key challenges in adapting Adaptive Feature Retention (AFR), an unstructured pruning technique, to structured pruning.
When applying AFR to structured pruning, three major problems arise: distribution mismatch between heterogeneous pruning scores, loss of sign information indicating optimization direction consistency, and influence of outliers. To address these issues, we propose a unified approach combining power transformation for nonlinear distribution alignment, sign-preserving score aggregation, and percentile-based outlier removal.
Experiments on Llama-3-8B, Vicuna-v1.5-13B, and LLaVA-v1.5-13B demonstrate that our method maintains accuracy comparable to unstructured pruning while achieving practical inference speedup through structured pruning.
Blogger's Review: This structured pruning method effectively tackles several challenges in traditional pruning techniques, particularly its significant contribution to enhancing inference efficiency while preserving model performance. As large language models become more prevalent, this approach holds great importance for optimizing resource utilization.