面向对话中多模态情绪预测与识别的情绪线索提取与融合
Emotional Cues Extraction and Fusion for Multi-modal Emotion Prediction and Recognition in Conversation
AI总结：
针对对话情绪预测中细粒度线索缺失与模态冗余问题，提出两阶段情绪线索提取与融合网络，利用词级标签和韵律学习构建模态情绪嵌入空间，并两步融合多模态特征，在IEMOCAP和MELD上取得优越性能。
摘要：
对话情绪预测（EPC）旨在利用前序对话来预测即将出现话语的情绪。以往的EPC方法依赖简单的上下文建模进行情绪提取，忽略了词级别的细粒度情绪线索。此外，先前工作未能考虑模态之间的内在差异，导致信息冗余。为克服这些局限，我们提出了一种情绪线索提取与融合网络，该网络包含两个阶段：模态特定学习阶段利用词级标签和韵律学习为每个模态构建情绪嵌入空间，以及用于整合多模态特征的两步融合阶段。此外，我们模型提取的情绪特征也适用于对话情绪识别（ERC）任务。实验结果验证了所提方法的有效性，在IEMOCAP和MELD数据集上均展现出优越性能。
英文摘要：
Emotion Prediction in Conversation (EPC) aims to forecast the emotions of forthcoming utterances by utilizing preceding dialogues. Previous EPC approaches relied on simple context modeling for emotion extraction, overlooking fine-grained emotion cues at the word level. Additionally, prior works failed to account for the intrinsic differences between modalities, resulting in redundant information. To overcome these limitations, we propose an emotional cues extraction and fusion network, which consists of two stages: a modality-specific learning stage that utilizes word-level labels and prosody learning to construct emotion embedding spaces for each modality, and a two-step fusion stage for integrating multi-modal features. Moreover, the emotion features extracted by our model are also applicable to the Emotion Recognition in Conversation (ERC) task. Experimental results validate the efficacy of the proposed method, demonstrating superior performance on both IEMOCAP and MELD datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
