Large language model (LLM) training is one of the fastest‑growing sources of electricity demand in modern data centers, and power availability is a primary bottleneck for AI infrastructure scaling. Making the power consumption of these workloads flexible could free additional electricity, curb price hikes, and improve grid utilization. To enable such flexibility we first need to understand how training throughput changes when GPU power is reduced. This paper provides the first systematic characterization of job power elasticity—the sensitivity of throughput to power reductions.
To quantify elasticity we introduce the Power Flexibility Index (PFI), defined as
$$ \text{PFI} = \frac{\frac{T{\text{reduced}}}{T{\text{full}}}}{\frac{P{\text{reduced}}}{P{\text{full}}}} $$
where $T$ is tokens processed per second and $P$ is the actual power draw. A PFI close to 1 indicates little throughput loss for a given power cut, while lower values signal poor elasticity.
We collected data from 131 LLM training runs on H200 GPUs (plus 24 H200 validation runs and 34 matched H100 runs), covering dense and mixture‑of‑experts models, pre‑training and fine‑tuning tasks, and up to 32 GPUs. Results show substantial but highly variable power elasticity across jobs; telemetry such as GPU utilization, memory occupancy, and kernel‑type distribution can predict PFI at runtime.
Using the predicted PFI, we implemented a PFI‑aware power allocation scheme that maximizes total tokens‑per‑second under a global power budget. With a 30% power reduction, the PFI‑aware allocator recovers roughly 1.5k tokens/s per job, closing about 63% of the gap between an equal‑weight allocation and an oracle with perfect information.
Our findings establish power elasticity as a measurable property of training jobs and lay the groundwork for power‑aware, grid‑responsive AI infrastructure.
Review