通过说话人识别、知识蒸馏和分层融合进行多说话人对话中的情绪识别
Emotion Recognition in Multi-Speaker Conversations through Speaker Identification, Knowledge Distillation, and Hierarchical Fusion
AI总结：
本文提出通过说话人识别、知识蒸馏和分层融合方法，提升多说话人对话中情绪识别的准确性和鲁棒性。
摘要：
多说话人对话中的情绪识别面临显著挑战，主要由于说话人模糊性和严重的类别不平衡问题。我们提出了一种新的框架，通过三个关键创新来解决这些问题：(1) 一个利用音频视觉同步来准确识别活跃说话人的说话人识别模块，(2) 一种将优越的文本情绪理解转移到音频和视觉模态的知识蒸馏策略，以及(3) 通过复合损失函数的分层注意力融合来处理类别不平衡。在MELD和IEMOCAP数据集上的全面评估显示了优越的性能，分别达到67.75%和72.44%的加权F1分数，特别是在少数情绪类别上取得了特别显著的改进。
英文摘要：
Emotion recognition in multi-speaker conversations faces significant challenges due to speaker ambiguity and severe class imbalance. We propose a novel framework that addresses these issues through three key innovations: (1) a speaker identification module that leverages audio-visual synchronization to accurately identify the active speaker, (2) a knowledge distillation strategy that transfers superior textual emotion understanding to audio and visual modalities, and (3) hierarchical attention fusion with composite loss functions to handle class imbalance. Comprehensive evaluations on MELD and IEMOCAP datasets demonstrate superior performance, achieving 67.75% and 72.44% weighted F1 scores respectively, with particularly notable improvements on minority emotion classes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
