Inference-time scaling is a promising paradigm to enhance generative models, particularly when outputs must meet structural constraints or optimize downstream rewards. We consider the Masked Diffusion Model (MDM) and introduce MDM-VGB, a discrete diffusion sampler that enhances unmasking generation with theoretically principled reward-guided remasking.
Inspired by the success of the classical Jerrum-Sinclair backtracking Markov chain in reward-tilted generation, MDM-VGB extends the backtracking random walk from a fixed prefix tree to a masked-state graph, allowing tokens to be unmasked and remasked at arbitrary positions. This sampler favors unmasking and remasking moves that lead to higher-value partial configurations, enabling effective high-reward generation and efficient repair of low-reward samples.
We prove that MDM-VGB is robust to process-verifier noise and achieves quadratic complexity, while popular test-time heuristics such as best-of-$N$ can incur exponential complexity due to error accumulation. Our theoretical findings are corroborated by strong empirical performance, particularly on popular constraint-satisfaction and scientific benchmarks such as Sudoku and QM9.
Blogger's Review: MDM-VGB's innovative application in masked diffusion models not only enhances generation quality but also reduces complexity, providing new insights for generative tasks under structural constraints. Its empirical success points to promising directions for future research.