The key‑value (KV) cache is a primary capacity and bandwidth bottleneck in long‑context LLM serving. We introduce Minima‑KV, a retention‑preserving hierarchy for mixed‑format paged attention. Recent and protected Anchor pages stay in FP8, while older non‑anchor pages are packed into TQ3; every live‑request page remains addressable. Format‑specific kernels compute partial attention states and merge them via a globally normalised online‑softmax, enabling direct heterogeneous decoding without a dense shadow of cache size.
On a single 96 GB NVIDIA RTX PRO 6000 Blackwell GPU, we evaluate various Qwen3.6‑27B configurations. Deployment reports only 18.3 KiB of attention KV per live token, a 3.50× compression versus BF16 and 1.75× versus FP8. Quality matches the dense baseline on 16 K RULER needle‑in‑a‑haystack tasks. On the same 503‑question LongBench v2 suite, the deltas at 16 K, 32 K and 64 K context lengths are –0.80, –0.60 and –0.40 percentage points respectively.
A separate single‑pair direct‑decode canary with two 59,008‑token requests measures a 3.625× active‑KV compression and 0.9821× throughput relative to its control, routes all 16 full‑attention layers without fallback, and retains no dense shadow. These results demonstrate that a mixed‑format path can practically compress long‑context state without evicting live‑request KV pages.
Blogger's Review: Minima‑KV achieves substantial KV compression while preserving decoding quality, marking a significant step forward for deploying long‑context LLMs.