Model compression makes large language models deployable, yet standard evaluation only reports task performance and misses how compression perturbs the distributed knowledge across billions of weights. ACE-Compression addresses this by shifting the question from “does the compressed model still answer correctly?” to “which of its core abilities remain intact?”. The framework probes four deployment‑facing dimensions: procedural reasoning and constraint execution, knowledge grounding and truthfulness, social cognition and affective understanding, and persona fidelity with stylistic consistency. Experiments on six open‑weight LLMs covering quantization and pruning yield over 35 compressed checkpoints. Results show that compression reshapes ability profiles: coarse, local, surface‑level signals tend to survive, while fine‑grained, long‑tail, and context‑dependent abilities become fragile. Quantization consistently retains more abilities than pruning, but the practical impact of this gap depends entirely on the abilities required by a given application. Consequently, deployers can select compression methods based on the ability profile their use case demands rather than a single leaderboard average, and algorithm designers gain diagnostic signals to build compression techniques that preserve what matters. ACE-Compression turns compression from a black‑box accuracy drop into an interpretable, measurable, and ultimately improvable intervention.
Review