Long-output reasoning makes the key‑value (KV) cache a critical memory bottleneck for efficient LLM serving. Existing KV compression techniques usually rely on a predefined per‑request budget and only decide which KV states to keep, leaving the total capacity unchanged throughout decoding. Yet reasoning workloads show large demand variation: different requests need different KV capacities, and the attention demand of a single request evolves during generation.
GrowPage treats KV capacity as a runtime resource and provides an on‑demand budgeting framework. It keeps lightweight dual‑timescale query summaries that capture recent and long‑term attention patterns, and uses the relative attention working sets to estimate demand evolution. At each capacity boundary, GrowPage either compresses KV states within the current allocation or acquires an additional physical page when broader demand emerges. By integrating with PagedAttention’s page‑level memory abstraction, GrowPage preserves continuous batching and prefix caching.
Experiments on reasoning benchmarks across several models demonstrate that GrowPage achieves a superior performance‑throughput trade‑off compared with prior approaches.
Review: GrowPage’s dynamic KV resource management alleviates memory pressure in long‑sequence inference, offering a more adaptable solution for large‑model serving.