Block diffusion has emerged as the dominant paradigm for scaling discrete diffusion language models (dLLMs), as decoding text in fixed-size blocks preserves parallel generation within each block while keeping quadratic attention costs manageable. However, this efficiency comes with a structural limitation: tokens near the end of a block are generated without access to future cross-block context, and once a block is finalized, its uncertain predictions become irreversible context for all subsequent blocks. This creates a block boundary problem, where uncertainty accumulates toward block boundaries and early mistakes propagate through later generations. To address this, we propose Multi-Block Editing (MBE) to mitigate this issue by editing decoded tokens based on cross-block context. MBE introduces a training-free decoding algorithm that edits decoded tokens in previous blocks by re-opening a full-attention window over selected blocks. Given the mismatched attention mechanism between block diffusion training and MBE inference, MBE also introduces a supervised fine-tuning strategy, providing the model with bidirectional attention masks that progressively expand the editing span. Furthermore, it extends SGLang with a multi-shape CUDA Graph pool and fine-grained KV cache control, making these variable-length editing passes efficient in practice. Experiments on LLaDA2.1-Mini across 13 benchmarks show that training-free MBE outperforms all existing decoding baselines while maintaining comparable throughput, with MBE SFT further yielding a performance gain of 2.7. The largest improvements are seen on tasks requiring strong long-range consistency, including +13.3 on AIME 2025 and +5.9 on ZebraLogic, demonstrating the effectiveness of MBE.
Blogger's Review: Multi-Block Editing (MBE) effectively addresses the block boundary issue through cross-block context editing, significantly enhancing model performance on long-range consistency tasks. Its training-free characteristic and efficient implementation provide fresh insights for future language model developments, especially in complex contextual scenarios. The practical application of this method warrants close attention, particularly in environments demanding intricate contextual processing.