Unified multimodal models aim to achieve any-to-any understanding and generation across arbitrary modalities. Existing approaches mainly rely on implicit statistical correlations and lack cross‑modal structural consistency constraints, which leads to semantic drift, poor compositional generalization, and instability under interventions.
This paper introduces C3-UniMM, a unified multimodal modeling framework built on Causal Cycle Consistency and Super Alignment. The key contributions are:
-
Structured Latent Causal Graph (SLCG): a shared cross‑modal semantic space that maps modalities onto a common causal structure.
-
Unified multimodal encoding blocks: jointly optimize understanding and generation within the identical causal semantic structure, ensuring causal consistency during encoding.
-
Unified Decoding Space: enforces structural preservation and semantic invertibility during cross‑modal generation, guaranteeing that generated outputs can be inversely mapped to the original semantics.
Theoretical analysis shows that SLCG and the unified decoding space markedly improve invertibility and mechanism invariance of cross‑modal mappings. Extensive experiments on understanding, generation, and compositional generalization tasks demonstrate that C3-UniMM consistently outperforms current unified multimodal baselines across metrics such as accuracy, BLEU, and CIDEr.
Blogger's Review: By grounding multimodal interactions in a causal graph, C3-UniMM offers a robust semantic bridge that mitigates long‑standing drift issues, making it a promising direction for real‑world multimodal applications.