从图谱视角重新审视对话中的多模态情感识别
Revisiting Multimodal Emotion Recognition in Conversation from the Perspective of Graph Spectrum
AI总结：
本文从图谱视角提出GS-MCC框架，通过滑动窗口构建多模态交互图并利用傅里叶图算子与对比学习协同提取高低频信息，以解决GNN在对话情感识别中难以捕获长距离一致性与互补性信息的问题，并在两个基准数据集上验证了其优越性。
摘要：
在多模态对话语境中高效捕获一致性与互补性语义特征对于对话中的多模态情感识别（MERC）至关重要。现有方法主要使用图结构对对话语境语义依赖进行建模，并采用图神经网络（GNN）捕获多模态语义特征以进行情感识别。然而，这些方法受限于GNN的一些固有特性，如过度平滑和低通滤波，导致无法高效学习长距离一致性信息和互补性信息。由于一致性和互补性信息分别对应低频和高频信息，本文从图谱的视角重新审视对话中的多模态情感识别问题。具体而言，我们提出了一个基于图谱的多模态一致性与互补性协同学习框架GS-MCC。首先，GS-MCC使用滑动窗口构建多模态交互图来建模对话关系，并利用高效的傅里叶图算子分别提取长距离高频和低频信息。然后，GS-MCC利用对比学习构建自监督信号，以反映高频与低频信号之间的互补性和一致性语义协同，从而提高高频和低频信息反映真实情感的能力。最后，GS-MCC将协同后的高频和低频信息输入MLP网络和softmax函数进行情感预测。大量实验证明了本文所提出的GS-MCC架构在两个基准数据集上的优越性。
英文摘要：
Efficiently capturing consistent and complementary semantic features in a multimodal conversation context is crucial for Multimodal Emotion Recognition in Conversation (MERC). Existing methods mainly use graph structures to model dialogue context semantic dependencies and employ Graph Neural Networks (GNN) to capture multimodal semantic features for emotion recognition. However, these methods are limited by some inherent characteristics of GNN, such as over-smoothing and low-pass filtering, resulting in the inability to learn long-distance consistency information and complementary information efficiently. Since consistency and complementarity information correspond to low-frequency and high-frequency information, respectively, this paper revisits the problem of multimodal emotion recognition in conversation from the perspective of the graph spectrum. Specifically, we propose a Graph-Spectrum-based Multimodal Consistency and Complementary collaborative learning framework GS-MCC. First, GS-MCC uses a sliding window to construct a multimodal interaction graph to model conversational relationships and uses efficient Fourier graph operators to extract long-distance high-frequency and low-frequency information, respectively. Then, GS-MCC uses contrastive learning to construct self-supervised signals that reflect complementarity and consistent semantic collaboration with high and low-frequency signals, thereby improving the ability of high and low-frequency information to reflect real emotions. Finally, GS-MCC inputs the collaborative high and low-frequency information into the MLP network and softmax function for emotion prediction. Extensive experiments have proven the superiority of the GS-MCC architecture proposed in this paper on two benchmark data sets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
