Abstract
DiLoCo-style training reduces communication by allowing learner islands to train locally before occasional outer synchronization, making it attractive for fragmented industrial AI fleets where training shares hardware with latency-sensitive serving. The critical question for such fleets is when an outer merge is worth its system cost and whether choosing which windows to defer matters at all.
Existing scheduling studies evaluate workload-aware policies against fixed-period baselines, but most omit the control that isolates timing from budget: matched random deferral, which inherits the controller's synchronization budget but is not itself deployable. This omission is consequential: across controlled stress tests and real vLLM sidecar replays, matched random ties or beats every forecast-free policy we test, so gains reported against weaker baselines cannot be attributed to window choice.
We fill this gap with Workload-Aware DiLoCo (WA-DiLoCo), a score-based controller that weighs learner progress against fleet pressure, and a calibration protocol that determines when matched random can be beaten, then demonstrate that it can. In the bursty regime where calibration exposes request-overlap structure, adding a one-step EWMA burst forecast to the online controller beats matched random in real vLLM sidecar replay, reducing SLO violations from 6.54% to 5.09% (8 of 10 seeds, $p=0.021$); offline Calibrated-WA, a non-deployable bound, shows the remaining headroom at 4.45% versus 6.26%. The deployable lesson remains the protocol: report real-sidecar effect-size transfer, a no-sync load match, and a matched-random envelope before claiming serving-SLO improvement.
Blogger's Review: This study introduces a novel approach to synchronization issues in shared AI infrastructure by implementing workload-aware scheduling and calibration protocols. It highlights the importance of optimizing resource utilization in complex systems, especially in latency-sensitive scenarios, which will be a key focus for future research.