NeFut Logo NeFut
Admin Login

[CS.AI] Vision Is Not Overhead: One-Pass Block Drafting for Lossless Speculative Decoding in Vision-Language Models

Published at: 2026-09-02 22:00 Last updated: 2026-09-03 02:56
#AI #Machine Learning #Open Source

Speculative decoding accelerates generation without altering its output, but on vision‑language models (VLMs) it falls into a self‑defeating loop. The drafter must stay autoregressive, thus remain small; a small drafter cannot afford to process the full image at every step, so the image is compressed, pruned, or hidden. Consequently, the drafter is least reliable exactly when the image makes the text most predictable. We introduce GLANCE, the first one‑pass block drafter that is lossless on an unmodified VLM target, breaking the loop at both ends. A block‑diffusion head reads the target’s already‑fused vision‑language state, so the drafter incurs no vision cost, and a single forward pass fills an entire block, eliminating sequential depth steps. A wide candidate tree is verified in one target pass, and every audited prompt reproduces greedy decoding exactly. Grounded tasks benefit the most, entering a verbatim‑copy regime where an autoregressive drafter needs a pass per token, while a block drafter needs only one per block. Under a single engine and one‑round budget, GLANCE decodes up to $2.93\times$ faster than autoregression, requiring one draft pass where the production EAGLE3‑VL head takes eight, and accepts blocks $2.7\times$ longer than an EAGLE‑3 head trained on the same corpus. Experiments show that accepted block length is set by the target’s next‑token entropy, with a fitted slope that steepens with grounding across all five tasks. This law transfers across targets and modalities and defines its own boundary, since free‑running text still favors a chain. Code is available at https://github.com/js-lee-AI/GLANCE

Review: GLANCE’s ability to read fused multimodal states in a single pass and generate whole blocks removes the vision overhead and sequential bottleneck, offering a practical and scalable solution for fast, lossless VLM generation.

Original Source: https://arxiv.org/abs/2609.00355

[h] Back to Home