Long‑context serving exhausts KV memory before hardware reaches its useful throughput. HiSparse stores history in host memory and fetches only selected entries, yet it allocates a fixed cache budget per load and cannot handle draft‑model speculative decoding such as DSpark and DFlash2. We introduce Adaptive DHiSparse, which enables speculative decoding over a hierarchical sparse KV and adapts residency to load. It protects the required working set, keeps extra history while GPU memory is free, and restores the rest via exact fused retrieval. DSpark and DFlash2 retain their proposal and acceptance rules: queries share missing‑row retrieval, draft KV uses a window‑bounded ring, and verification staging is reused across layer groups. At 100K context and concurrency 128, GLM‑5.3 with DFlash2 delivers 793 output tokens per second per decode GPU, a 1.45× gain over HiSparse and 1.27× over non‑speculative adaptive serving. Latency stays within HiSparse bounds, while concurrency scales up to 4× and output increases 2.3‑3.7× depending on the per‑token bound. On DeepSeek‑V4.1‑Flash at 500K context and concurrency 512, Adaptive DHiSparse with DSpark yields 1.85× the output of resident serving with DSpark. During a load step, optional residency cuts host fetches from 14.2% to 1.4% of selected rows at low load and releases memory under pressure.
Review