In today's computing environments, Large Language Models (LLMs) are widely adopted due to their strong reasoning and query-response capabilities. However, deploying them in embedded and edge computing environments poses strict latency, memory, and energy constraints. The large parameter counts and computational demands of LLMs hinder efficient execution on resource-constrained platforms. While model pruning has emerged as a viable solution to reduce scale while preserving performance, jointly optimizing layers, attention heads, and Multi-Layer Perceptron (MLP) dimensions remains highly complex. Exhaustively exploring this combined design space is computationally expensive and often leads to local optima or unstable configurations.
To address these limitations, we propose a hardware-aware, multi-objective structured pruning framework. This proposed two-stage method explicitly targets latency and model size for efficient deployment on edge devices. In the coarse-grained stage, multi-objective depth pruning removes entire attention and MLP blocks to reduce computational load and memory usage. In the subsequent fine-grained stage, Parallel Bayesian Optimization (PBO) searches for the optimal layer-wise pruning ratios under latency constraints, while importance-based strategies rank specific components to be pruned within each layer's allocated budget. Experimental results show that our approach reduces model complexity with minimal impact on commonsense reasoning tasks and zero-shot performance. Our method achieves a favorable trade-off among accuracy, latency, and model size, making it suitable for edge deployment. Across multiple LLMs at pruning ratios of 37.5% and 50%, the proposed approach outperforms existing methods on commonsense reasoning tasks while significantly reducing inference costs.
Blogger's Review: This research provides an effective solution for optimizing LLMs in resource-constrained environments, achieving a good balance between latency and model size. The multi-stage optimization strategy and parallel Bayesian optimization method employed are worth further exploration and application in future research.