Multi‑Head Latent Attention (MLA) has become a core component of recent frontier large language models. Unlike standard attention that caches per‑head keys and values, MLA stores a compressed KV latent tensor that is up‑projected into keys and values at decode time, together with a narrow positional key that injects position information into the attention scores. Existing KV‑cache quantization schemes are built for standard attention and ignore the actual structure of MLA caches, causing significant accuracy loss at low bit widths.
We analyze the MLA cache from both activation distribution and quantization‑error propagation perspectives, revealing that the two cached tensors play distinct roles: the latent mainly supplies the values, while the positional key dominates the attention scores. Accordingly, we propose RAQ‑MLA (Role‑Aware Quantization for Multi‑Head Latent Attention Caches), which fits an invertible transform to each cached tensor to minimize the error it contributes to the attention output—attention‑weighted value error $\sum_i \alpha_i \Delta V_i$ for the latent and score perturbation for the positional key.
Without any additional training, RAQ‑MLA quantizes the full MLA cache to roughly 2.5 bits per element, a 6.5× increase in theoretical KV‑cache capacity, while remaining nearly lossless on reasoning and coding tasks across four MLA models.
Review