通过跨模态注意力对齐和平衡堆叠模型改进语音情感识别
Improving Speech Emotion Recognition Through Cross Modal Attention Alignment and Balanced Stacking Model
AI总结：
针对自然语音情感识别数据不平衡问题，提出利用跨模态注意力融合多模态表示，并结合加权交叉熵损失与软间隔损失训练12个模型进行平衡堆叠，在8类识别任务上取得显著效果。
摘要：
情感在人类交互中起着基础作用，因此能够在语音中识别情感的系统在人机交互背景下至关重要。语音情感识别（SER）是一个具有挑战性的问题，特别是在自然语音以及可用数据在不同情感间不平衡的情况下。本文提出了我们在2025年自然条件语音情感识别挑战赛背景下提出的系统。我们提出的架构利用跨模态，使用跨模态注意力来融合来自不同模态的表示。为了解决类别不平衡问题，我们采用了两种训练设计：（i）加权交叉熵损失（WCE）；以及（ii）WCE加上额外的中性-表达性软间隔损失和平衡。我们共训练了12个多模态模型，并使用平衡堆叠模型进行集成。我们提出的系统在8类语音情感识别上实现了0.4094的Macro-F1分数和0.4128的准确率。
英文摘要：
Emotion plays a fundamental role in human interaction, and therefore systems capable of identifying emotions in speech are crucial in the context of human-computer interaction. Speech emotion recognition (SER) is a challenging problem, particularly in natural speech and when the available data is imbalanced across emotions. This paper presents our proposed system in the context of the 2025 Speech Emotion Recognition in Naturalistic Conditions Challenge. Our proposed architecture leverages cross-modality, utilizing cross-modal attention to fuse representations from different modalities. To address class imbalance, we employed two training designs: (i) weighted crossentropy loss (WCE); and (ii) WCE with an additional neutralexpressive soft margin loss and balancing. We trained a total of 12 multimodal models, which were ensembled using a balanced stacking model. Our proposed system achieves a MacroF1 score of 0.4094 and an accuracy of 0.4128 on 8-class speech emotion recognition.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
