Abstract
Multimodal large language models (MLLMs) can connect visualization patterns to external causes, consequences, and domain knowledge, but the evidential basis of these interpretations is often unclear.
We present an exploratory study of 102 visualizations from four sources, three MLLMs, and four input conditions that vary access to the image, source-specific accessible chart context, and withheld-context framing.
Across 1,224 descriptions, we analyze model-attributed DIRECT, DERIVED, and SPECULATIVE labels and conduct an automated audit of numeric agreement. Accessible chart context shifted Gemini and GPT toward DIRECT claims and improved numeric agreement for some models.
Adding the image to the full context did not yield a consistent numeric benefit, and the withheld-context prompt did not reliably increase cautious language. The prompt-defined Real-World Significance section remained predominantly SPECULATIVE.
These results motivate accessible description systems that distinguish claims supported by supplied evidence from model-supplied interpretation.
Blogger's Review: This paper provides an in-depth analysis of the application of multimodal large language models in visualization generation, revealing the complexity of model-generated claims and their performance variations across different contexts. It emphasizes the necessity of developing verifiable visualization description systems, offering significant direction and insights for future model applications in the visualization domain.