Diffusion‑based multimodal large language models (dMLLMs) decode by iterative denoising with bidirectional attention, which prevents the use of the standard autoregressive key‑value cache. In practice, approximate caching is employed: intermediate states are reused across denoising steps. This caching often causes a token to repeat at adjacent positions, breaking sentence fluency—a failure we call the Repeat Curse.
Tracing the information flow reveals three disruptions:
- Within‑step disruption – caching disperses attention, so a position no longer anchors tightly on nearby context tokens;
- Across‑step disruption – similarity‑based refresh is not staleness‑aware, causing some positions to be decoded against stale KV states;
- Across‑layer disruption – deep‑layer context entropy fails to converge, and high‑entropy layers tend to produce repetitions.
We therefore propose CoTA++, a training‑free, plug‑and‑play framework that counteracts each disruption at its source:
- Context‑Token Attention Re‑anchoring rebuilds attention routing against the current context, restoring within‑step anchoring;
- Decode‑Aware Refresh reserves part of the cache’s refresh budget for tokens about to be committed, avoiding stale KV usage;
- Context‑Token Entropy‑Guided Voting discounts a position’s confidence by the deep‑layer entropy of its context and reorders the unmasking accordingly.
The three components operate at non‑overlapping stages of a decoding step and can be freely composed with existing dMLLMs and caches. Across three dMLLMs and two caching backends, CoTA++ reduces the sample repetition rate of long responses from up to 99.6% to at most 17.0%, recovers most of the accuracy lost to caching on nine benchmarks, and keeps the speedup overhead under 15%, making approximate caching reliable for dMLLMs.
Review: CoTA++ offers a fine‑grained information‑flow intervention that effectively mitigates the inherent flaws of iterative denoising caches, delivering a practical reliability boost for large‑scale multimodal generation.