Internet memes intertwine visual cues, textual content, and cultural context, making them particularly challenging to interpret in scenarios where humor, sarcasm, and harmful intent coexist. These complexities highlight the need for explainable meme understanding systems that can provide reliable and structured reasoning to support both accurate classification and human interpretability. However, existing multimodal classifiers either overlook these interdependencies or provide only limited interpretability.
This paper introduces MAR-12, a novel framework leveraging Vision Language Models (VLMs) for meme detection and understanding in settings where humorous and hateful elements may coexist. The framework first interprets each meme through twelve structured perspectives derived from humor and hate theories. It then applies a role-aware soft-gated attention mechanism to learn the contribution of each perspective, followed by a prototype-based classifier for final prediction.
Explanations are synthesized using both perspective-specific reasoning and learned attention weights, ensuring transparent and context-grounded justifications. We evaluate MAR-12 on the PrideMM and Memotion datasets, achieving up to 80.3% accuracy for humor detection and 75.9% for hate detection, outperforming state-of-the-art approaches. Furthermore, both human and GPT-4-based evaluations confirm that MAR-12 produces coherent and persuasive explanations, especially for memes where humorous and harmful cues co-occur.
Blogger's Review: The MAR-12 framework effectively addresses the complexities of humor and hate content through a multi-angle reasoning mechanism, providing a new tool for understanding and analyzing meme culture. This research not only enhances meme detection accuracy but also offers crucial insights into the cultural context behind them, making it a valuable addition to broader text analysis applications.