As large language models (LLMs) reshape software development and maintenance, inference engines like vLLM are efficiently serving pre-trained and highly configurable models. While prior research has focused on model architectures and hardware acceleration, the impact of inference engine configuration on energy consumption, performance, and output quality remains poorly understood.
This study presents a large-scale controlled experiment analyzing three vLLM configuration options: attention kernel type, prefix caching, and chunked prefill. We evaluated all combinations of these configurations across 5 open-weight LLMs and 5 diverse inference tasks, totaling $9,000$ runs and $93,600$ measures.
Our results indicate that the studied configuration options significantly affect energy consumption and performance, primarily driven by attention type and prefix caching, while chunked prefill shows limited effects under the default vLLM serving configuration and evaluated workloads. These effects are highly model- and workload-dependent, and no single configuration is universally optimal. We further demonstrate that model choice dominates global trade-offs, while configuration tuning provides local improvements along the Pareto frontier. Unexpectedly, inference options can also impact model accuracy.
Blogger's Review: This paper systematically investigates the effects of vLLM configurations on energy and performance, highlighting the potential for local optimizations across different models and tasks, providing valuable insights for practical applications.