arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2509.20373cs.CLcs.LG

说话人风格感知的音素锚定用于改进跨语言语音情感识别

Speaker Style-Aware Phoneme Anchoring for Improved Cross-Lingual Speech Emotion Recognition

Department of Electrical Engineering, National Tsing Hua University, Taiwan(National Tsing Hua University电子工程系);Language Technologies Institute, Carnegie Mellon University, USA(卡内基梅隆大学语言技术研究所)

Shreya G. Upadhyay, Carlos Busso, Chi-Chun Lee

AI总结:

针对跨语言语音情感识别中语音差异和说话人风格差异的挑战,提出说话人风格感知的音素锚定框架,通过图聚类和双空间锚定提升跨语言情感迁移,并在英语和台湾普通话语料库上验证了其优越性。

摘要:

跨语言语音情感识别(SER)由于不同语言在语音变异性和说话人特定表达风格上的差异,仍然是一项具有挑战性的任务。在这种多样化条件下有效捕捉情感,需要一个能够跨不同说话人和语言对齐情感外在表现的框架。为解决这一问题,我们提出了一种说话人风格感知的音素锚定框架,该框架在音素层面和说话人层面同时对情感表达进行对齐。我们的方法通过基于图的聚类构建情感特定的说话人社区,以捕捉共享的说话人特质。利用这些群体,我们在说话人空间和音素空间中应用双空间锚定,以实现更好的跨语言情感迁移。在MSP-Podcast(英语)和BIIC-Podcast(台湾普通话)语料库上的评估表明,与竞争性基线相比,我们的方法具有更好的泛化能力,并为跨语言情感表征的共性提供了有价值的见解。

英文摘要:

Cross-lingual speech emotion recognition (SER) remains a challenging task due to differences in phonetic variability and speaker-specific expressive styles across languages. Effectively capturing emotion under such diverse conditions requires a framework that can align the externalization of emotions across different speakers and languages. To address this problem, we propose a speaker-style aware phoneme anchoring framework that aligns emotional expression at the phonetic and speaker levels. Our method builds emotion-specific speaker communities via graph-based clustering to capture shared speaker traits. Using these groups, we apply dual-space anchoring in speaker and phonetic spaces to enable better emotion transfer across languages. Evaluations on the MSP-Podcast (English) and BIIC-Podcast (Taiwanese Mandarin) corpora demonstrate improved generalization over competitive baselines and provide valuable insights into the commonalities in cross-lingual emotion representation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑