In RL-based large language model (LLM) post-training, Rollout and Training are typically allocated to separate GPU resources, but static GPU partitioning suffers from severe pipeline bubbles under long-tail rollout latency.
To address this, we present DynaResize, a dynamic GPU reallocation system that switches GPUs between Rollout and Training at runtime to balance stage execution times without changing RL semantics.
DynaResize decomposes resizing into fine-grained operations and removes non-startup-critical work from the critical path through communicator reuse, bounded state staging, and hysteresis-based resizing.
Experimental results show that DynaResize can improve end-to-end throughput by 66.5% and reduce total execution time by 33% over the optimal static configuration, while hiding 27% of role-switching overhead.
Blogger's Review: DynaResize significantly enhances the efficiency of post-training for large-scale models through dynamic GPU resource allocation, showcasing innovative potential in resource management. This method not only boosts throughput but also effectively reduces execution time, making it worth promoting in broader deep learning tasks.