用于对话情感识别的 Mamba 增强型文本-音频-视频对齐网络
Mamba-Enhanced Text-Audio-Video Alignment Network for Emotion Recognition in Conversations
AI总结：
针对对话情感识别中的多模态一致性与长程上下文建模难题，提出 MaTAV 网络，通过 Mamba 对齐文本、音频和视频特征并处理长序列，在 MELD 与 IEMOCAP 上显著超越现有方法。
摘要：
对话情感识别（ERC）是多模态交互研究中的一个重要领域，致力于准确识别并分类说话者在整个对话过程中表达的情感。传统 ERC 方法主要依赖单模态线索——例如文本、音频或视觉数据——导致其有效性存在局限。这些方法面临两个重大挑战：1）多模态信息的一致性。在整合各种模态之前，必须确保来自不同来源的数据彼此对齐且连贯。2）上下文信息捕获。要成功融合多模态特征，需要敏锐理解不断变化的情感基调，尤其是在情感可能随时间转变和发展的长篇对话中。为解决这些局限，我们针对 ERC 任务提出了一种新颖的 Mamba 增强型文本-音频-视频对齐网络（MaTAV）。MaTAV 的优势在于能够对齐单模态特征以确保不同模态间的一致性，并能处理长输入序列，从而更好地捕获上下文多模态信息。在 MELD 和 IEMOCAP 数据集上进行的大量实验表明，MaTAV 在 ERC 任务上以大幅优势显著优于现有的最先进方法。
英文摘要：
Emotion Recognition in Conversations (ERCs) is a vital area within multimodal interaction research, dedicated to accurately identifying and classifying the emotions expressed by speakers throughout a conversation. Traditional ERC approaches predominantly rely on unimodal cues\-such as text, audio, or visual data\-leading to limitations in their effectiveness. These methods encounter two significant challenges: 1) Consistency in multimodal information. Before integrating various modalities, it is crucial to ensure that the data from different sources is aligned and coherent. 2) Contextual information capture. Successfully fusing multimodal features requires a keen understanding of the evolving emotional tone, especially in lengthy dialogues where emotions may shift and develop over time. To address these limitations, we propose a novel Mamba-enhanced Text-Audio-Video alignment network (MaTAV) for the ERC task. MaTAV is with the advantages of aligning unimodal features to ensure consistency across different modalities and handling long input sequences to better capture contextual multimodal information. The extensive experiments on the MELD and IEMOCAP datasets demonstrate that MaTAV significantly outperforms existing state-of-the-art methods on the ERC task with a big margin.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
