We directly measured inference energy consumption for 86 open‑weight model checkpoints across seven benchmarks and six hardware platforms, ranging from a consumer GPU to an eight‑GPU server, to quantify the trade‑off between capability and energy use. The results reveal huge efficiency gaps even among models with similar performance. Most configurations lie far from the score‑energy Pareto frontier, consuming a median of $2.6\sim4.8$ times more energy than another configuration that scores at least as well, with extreme differences reaching $27$ times.
Reasoning modes have a pronounced impact. Enabling chain‑of‑thought reasoning can improve capability but raises energy consumption by about $3.7$ times on average, and the benefit varies strongly with model family and output budget. Token count is an imperfect proxy for energy, misordering $18\sim24\%$ of model pairs.
At the hardware level, FP8 arithmetic reduces energy by roughly a quarter with little change in benchmark scores. Switching hardware mainly rescales energy consumption without altering the relative ranking of models.
These findings demonstrate that capability scores alone provide an incomplete picture of efficiency. We argue that benchmark results should be reported together with measured energy consumption under a defined serving regime and output budget, enabling a more comprehensive assessment of real‑world model costs.
Review