Speculative decoding speeds up autoregressive generation by letting a lightweight drafter propose future tokens while the target model verifies them in parallel. Its lossless guarantee has driven drafter models toward parallel generation. The latest block‑parallel generative drafting paradigm, including diffusion‑based methods such as DFlash and DSpark, achieves up to 3.6× speedup on everyday chat tasks. While text‑only LLMs have been extensively studied, the applicability of this technique to multimodal models remains unclear. Existing multimodal speculative decoding work focuses on input compression, adapter alignment, candidate coverage, or modality‑specific verification, leaving block‑parallel drafting largely unexplored.
To bridge this gap, this paper first conducts a modality‑centered survey and then performs a cross‑architecture empirical study, asking: Is multimodal speculative decoding ready for diffusion‑based parallel drafting? We systematically analyze Vision‑Language, Video‑Language, Audio, and Vision‑Language‑Action models from the dual perspectives of drafting parallelism and cross‑modal information interaction. A unified taxonomy is introduced that separates drafter‑side parallelism from orthogonal design choices such as tree construction and verification strategies. We then provide a comprehensive empirical comparison of existing methods under varying degrees of parallelism on standardized multimodal benchmarks, including OCR, VQA, visual reasoning, and image captioning.
Finally, we summarize the limitations of current approaches, discuss open challenges, and outline promising future directions.
Blogger's Review: The survey offers a clear taxonomy and solid empirical evidence, highlighting both the potential and the hurdles of applying diffusion‑based parallel drafting to multimodal models.