Multimodal large language models (MLLMs) have greatly advanced multimodal emotion recognition (MER) by enabling fine‑grained emotion understanding through reasoning over multimodal evidence. Yet, they still suffer from systematic confusion between semantically adjacent emotions whose cues heavily overlap. Existing solutions typically rely on global parameter adaptation, which is costly and inflexible. We notice that emotion confusion is highly localized, suggesting that reusable pair‑specific expertise is a better remedy.
To this end, we introduce Confusion‑Aware Skill Evolution (CASE), the first agent‑skill framework for MER. CASE maintains a sparse, self‑evolving library of EmotionSkills, each encapsulating expertise for a specific emotion pair. By externalizing affective knowledge from the model parameters, these skills can be continuously refined and readily transferred across diverse frozen MLLM backbones. During inference, CASE dynamically retrieves the appropriate EmotionSkill to reinforce boundary‑aware reasoning while keeping the MLLM backbone frozen.
Extensive experiments on CA‑MER, DFEW, and EMER demonstrate CASE’s effectiveness and transferability for both general‑purpose and emotion‑specialized MLLMs. Notably, CASE lifts the average accuracy of Qwen3‑VL‑8B on the CA‑MER benchmark from 46.80% to 59.13%, and yields consistent gains across other models.
Review: CASE’s modular, evolving skill library offers a cost‑effective and adaptable solution to emotion confusion, opening new avenues for enhancing frozen multimodal models in emotion recognition tasks.