Recent Omni LLMs (OLLMs) have shown strong audio‑visual reasoning abilities, yet cross‑modal hallucinations remain a critical issue. Existing mitigation techniques mainly intervene at the output stage, overlooking the entanglement of grounding evidence and learned priors within fused representations. To tackle this, we introduce Omni‑Woodpecker (OWP), a training‑free framework that shifts hallucination correction from output manipulation to conflict‑conditioned representation rebalancing.
OWP first detects cross‑modal conflicts by comparing complementary modality‑specific views. It then applies orthogonal decomposition to separate grounding evidence from competing priors in the hidden states. Finally, before decoding, OWP adaptively suppresses misleading priors and restores ignored evidence, achieving a balanced fused representation.
Extensive experiments using Qwen2.5‑Omni‑7B as the backbone show OWP improves scores by 12.9 and 8.6 points on CMM and AVHBench respectively, and yields consistent gains of 0.67‑2.25 points across four general OLLM reasoning benchmarks. The method also generalizes well to VideoLLaMA2‑AV, adding 4.6 and 3.9 points on the same benchmarks. Code is publicly available at https://github.com/ZiiJianLiu/Omni-Woodpecker.
Review