EmoEUS：对话中多模态情感识别的不确定性监督
EmoEUS: Uncertainty Supervision for Multimodal Emotion Recognition in Conversation
浏览论文内容
中文总结 AI 辅助
研究对话中多模态情感识别，提出EmoEUS框架，通过动态加权模态执行不确定性感知多模态融合，并引入显式监督损失，实验证明该框架优于现有方法。
中文摘要 AI 辅助
对话中的多模态情感识别（MERC）可利用多模态和上下文线索提高识别性能。但现有融合方法常忽略因线索冲突、噪声变化和模态特定信号缺失导致的跨话语模态特定不确定性。我们提出EmoEUS，一个用于MERC的显式不确定性监督框架。它通过使用学习到的方差估计动态加权模态来执行不确定性感知多模态融合。我们还引入了一个显式监督损失，使每个话语的预测方差与话语的分布表示与其情感和模态特定聚类中心之间的距离对齐。在IEMOCAP和MELD上的实验表明，EmoEUS始终优于现有方法。
英文摘要
Multimodal emotion recognition in conversation (MERC) can leverage multimodal and contextual cues to boost recognition performance. However, existing fusion approaches in MERC often ignore modality-specific uncertainty across utterances caused by conflicting cues, varying noise, and missing modality-specific signals. We propose EmoEUS, an explicit uncertainty supervision framework for MERC. EmoEUS performs uncertainty-aware multimodal fusion by dynamically weighting modalities using learned variance estimates. We also introduce an explicitly supervised loss that aligns each utterance's predicted variance with the distance between the utterance's distributional representation and its emotion- and modality-specific cluster center. Experiments on IEMOCAP and MELD show that EmoEUS consistently outperforms state-of-the-art methods.
发表机构
- Dept. of Electrical and Electronic Engineering, The Hong Kong Polytechnic University（香港理工大学电子与电气工程系）
- Speech, Language, and Cognition Laboratory, The University of Hong Kong（香港大学语音、语言与认知实验室）
机构由 AI 辅助整理，请以论文原文为准。
