Existing text summarization research has focused primarily on monologic information (e.g., newspaper articles, reports), overlooking the interaction between speakers or authors. In contrast, dialogues serve as a rich communication channel where multiple participants engage in back-and-forth exchanges to construct meaning. We propose a dialogue summarization framework that explicitly models both semantic and emotion dynamics using multimodal dialogue inputs, built on an adapted hierarchical Chain-of-Agents approach.
Our framework decomposes dialogues from two perspectives:
- Topic segments based on the utterances of all participants;
- Participant-specific utterance segments.
These segments are used to generate corresponding summaries while incorporating automatically inferred emotions. Topic- and participant-level summaries are aggregated into a dialogue summary that captures semantic content and emotional trajectories. To evaluate beyond content accuracy, we introduce emotion trajectory metrics to measure how well summaries preserve emotional flow.
Experiments with small language models on multimodal dialogue datasets demonstrate that our framework produces summaries containing both semantic and emotional content. Further experiments on explicit emotion label availability highlight the efficacy of our proposed methodology and the opportunities in dialogue analysis using language models.
Blogger's Review: This study innovatively incorporates emotion dynamics into the dialogue summarization framework, focusing not only on content accuracy but also on the preservation of emotional flow. This approach significantly enhances the depth and multidimensionality of summaries, providing new perspectives and rich research opportunities in dialogue analysis, especially in the application of multimodal data.