In in-context learning (ICL), multimodal large language models (MLLMs) can classify images from a few labeled examples. However, how these models utilize the provided context remains unclear. While Chain-of-Thought prompting is widely adopted, recent studies argue that it may not reflect true internal computation.
This paper systematically evaluates the concept-based explainability of frozen MLLMs under few-shot ICL using five conditions of increasing formal rigor, from baseline classification to Description Logics (DL) axiom generation. By evaluating four state-of-the-art MLLMs through an independent LLM-as-a-judge pipeline, we demonstrate that explaining is genuinely harder than predicting alone.
Surprisingly, forcing models to generate formally structured, concept-based explanations results in a monotonically decreasing predictive accuracy (from 93.8% to 90.1%), contradicting the assumption that explicit reasoning universally aids performance. However, when models successfully articulate class-discriminative visual features, the quality of explanations strongly correlates with correct predictions.
Our findings suggest that while MLLMs excel at visual classification, they lack the specific instruction-tuning necessary for formal, machine-verifiable explainability.
Blogger's Review: This research reveals the potential limitations of MLLMs in handling visual classification, particularly in their explanatory capabilities. Although the models perform well in predictions, the lack of explainability may hinder their effectiveness in practical applications, highlighting the need for further exploration on optimizing instruction tuning to enhance explainability.