arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.18010eess.AScs.SD
收藏

AmbER$^2$: 双重模糊感知情感识别应用于语音和文本

AmbER$^2$: Dual Ambiguity-Aware Emotion Recognition Applied to Speech and Text

Massachusetts Institute of Technology(麻省理工学院)

Jingyao Wu, Grace Lin, Yinuo Song, Rosalind Picard

AI总结:

AmbER$^2$通过双重模糊感知框架提升语音和文本情感识别的准确性与鲁棒性。

摘要:

情感识别本质上具有模糊性,不确定性来源于评分者之间的分歧以及不同模态(如语音和文本)之间的差异。近年来,利用标签分布建模评分者模糊性引起了广泛关注。然而,模态模糊性仍处于探索阶段,多模态方法通常依赖于简单的特征融合,而没有明确处理模态之间的冲突。在本工作中,我们提出AmbER$^2$,一个双重模糊感知框架,通过具有分布性训练目标的教师-学生架构,同时建模评分者级别和模态级别模糊性。在IEMOCAP和MSP-Podcast上的评估表明,AmbER$^2$在传统交叉熵基线上始终提高了分布忠实度,并且在性能上与或优于最近的最先进系统相竞争。例如,在IEMOCAP上,AmbER$^2$在Bhattacharyya系数上实现了20.3%的相对提升(0.83 vs. 0.69),在R$^2$上实现了13.6%的提升(0.67 vs. 0.59),在准确率上实现了3.8%的提升(0.683 vs. 0.658),在F1上实现了4.5%的提升(0.675 vs. 0.646)。进一步分析不同模糊级别显示,显式建模模糊性对高不确定样本特别有益。这些发现突显了在构建稳健的情感识别系统时共同处理评分者和模态模糊性的重要性。

英文摘要:

Emotion recognition is inherently ambiguous, with uncertainty arising both from rater disagreement and from discrepancies across modalities such as speech and text. There is growing interest in modeling rater ambiguity using label distributions. However, modality ambiguity remains underexplored, and multimodal approaches often rely on simple feature fusion without explicitly addressing conflicts between modalities. In this work, we propose AmbER$^2$, a dual ambiguity-aware framework that simultaneously models rater-level and modality-level ambiguity through a teacher-student architecture with a distribution-wise training objective. Evaluations on IEMOCAP and MSP-Podcast show that AmbER$^2$ consistently improves distributional fidelity over conventional cross-entropy baselines and achieves performance competitive with, or superior to, recent state-of-the-art systems. For example, on IEMOCAP, AmbER$^2$ achieves relative improvements of 20.3% on Bhattacharyya coefficient (0.83 vs. 0.69), 13.6% on R$^2$ (0.67 vs. 0.59), 3.8% on accuracy (0.683 vs. 0.658), and 4.5% on F1 (0.675 vs. 0.646). Further analysis across ambiguity levels shows that explicitly modeling ambiguity is particularly beneficial for highly uncertain samples. These findings highlight the importance of jointly addressing rater and modality ambiguity when building robust emotion recognition systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑