NeFut Logo NeFut
Admin Login

[CS.AI] Trajectory-Aware Decoding Control for Diffusion Vision-Language Models

Published at: 2026-09-12 22:00 Last updated: 2026-09-15 01:15
#AI #Machine Learning #Neural

Diffusion vision-language models generate answers by iterative refinement, leaving intermediate answer trajectories that can be inspected and steered at inference time. Applying a single generation length to all questions creates a reasoning-budget mismatch: visually closed questions suffer when refinement continues after the answer stabilizes, while reasoning-intensive questions are harmed by premature commitment. We name this problem reasoning-budget mismatch and study it on LLaDA-V. Instead of a universal length, we introduce a training-free controller that routes each instance to early commitment, baseline preservation, or reasoning-supportive decoding. The routing relies on trajectory signals such as answer closure, commitment evidence, and representation revision pressure, without using ground-truth answers. Across answer-focused, mixed-reasoning, and chain-of-thought-sensitive benchmarks, the routed control outperforms fixed long decoding, pure short decoding, and single-rule interventions, and the gains are not explained by shorter outputs alone. Answer-closed examples benefit from early commitment, whereas chain-of-thought examples require preserving or supporting intermediate reasoning. These findings suggest that diffusion VLM decoding should be guided by the state indicated by the observed trajectory rather than a universal decoding length.

Review: This study shows that trajectory-aware routing can adapt decoding to diverse reasoning demands, offering a practical, training-free way to improve robustness of diffusion-based vision-language models.

Original Source: https://arxiv.org/abs/2609.11315

[h] Back to Home