Abstract
Diffusion-based Large Language Models (DLLMs) enable parallel generation via Semi-Autoregressive (SAR) decoding in text generation. However, current methods suffer from severe operator-level redundancy: they recompute the entire sequence during denoising steps, ignoring that the prefix and masked suffix remain invariant within a block. We propose LaCache, a training-free acceleration framework that alleviates this redundancy through lossless caching and mixed precision.
Specifically, LaCache employs Lossless State Memoization (LSM) by caching three types of intermediate results:
- EmbedCache for embedding outputs,
- RoPECache for token-wise pre-attention states, and
- FACache for the online softmax statistics within FlashAttention.
These caches allow the model to skip redundant computation on unchanged tokens without altering the output. To further alleviate memory-bandwidth bottlenecks, LaCache integrates a per-group FP8 quantization strategy for FFN layers, tailored to step-dependent activation distributions across the diffusion process.
Experiments demonstrate that LaCache alone achieves approximately 1.3X end-to-end speedup over vanilla DLLM. When combined with existing acceleration methods, LaCache reaches up to 40.2X end-to-end speedup while maintaining comparable task accuracy.
Blogger's Review: The introduction of LaCache not only effectively addresses the redundancy in computation for current DLLMs but also achieves significant performance improvements through lossless caching and mixed precision strategies. This presents a new perspective for efficient inference in large language models, making it worthy of attention and further exploration.