[CS.AI] Symbal: Detecting Systematic Misalignments in Model-Generated Captions

Abstract

Multimodal large language models (MLLMs) often introduce errors when generating image captions, resulting in misaligned image-text pairs. Our work focuses on a class of captioning errors we refer to as systematic misalignments, where a recurring error in MLLM-generated captions is closely associated with the presence of a specific visual feature in the paired image.

Research Goals

Given a vision-language dataset with MLLM-generated captions, our aim is to detect such errors, a task we refer to as systematic misalignment detection.

Key Contributions

1. Symbal: We present a structured, dual-stage setup utilizing off-the-shelf foundation models to identify systematic misalignments and summarize results in natural language.
2. SymbalBench: We introduce a benchmark designed to evaluate automated methods on our proposed task, consisting of 1.7 million image-text pairs from two domains (natural and medical images) organized into 420 vision-language datasets with annotated systematic misalignments.

Performance Evaluation

Symbal exhibits strong performance on this benchmark, correctly identifying systematic misalignments in 63.8% of datasets, nearly 4x improvement over the closest baseline. We supplement evaluations on SymbalBench with real-world assessments showing that (1) Symbal can accurately surface systematic misalignments in captions generated by four MLLMs and (2) Symbal is a powerful tool for auditing off-the-shelf image-caption datasets.

Ultimately, our novel task, method, and benchmark can aid users in auditing MLLM-generated captions and identifying critical errors without requiring access to the underlying MLLM. Code is available at GitHub.

Blogger's Review: By introducing Symbal and SymbalBench, the researchers provide an effective solution for detecting systematic misalignments, enhancing the quality of image-text alignment and offering new insights for auditing in multimodal learning. The in-depth analysis of visual features will drive the accuracy of model-generated content, showcasing broad application prospects.