语音强调模型能否跨语言和情感泛化?
Do Speech Emphasis Models Generalize across Languages and Emotions?
AI总结:
本文构建多语言多情感强调语料库MMEE,通过跨语言、跨情感等实验发现单语模型零样本迁移有限,而多语言训练显著提升鲁棒性。
摘要:
韵律强调因语言、情感和说话风格而异,然而现有的强调检测模型主要是在单语中性朗读语音上训练和评估的。我们引入了MMEE(多语言多情感强调),这是一个包含10,000句专业录制的表达性话语(14.13小时)的语料库,涵盖7种语言和34种情感/风格类别,并带有三级感知标签(每个样本10个标注)。我们在单语、跨语言、多语言、跨情感、跨数据集和数据规模设置下对两种最先进的架构进行了基准测试。单语模型显示出有限的零样本迁移能力,在类型学上相距较远的语言上性能下降,而多语言训练显著提高了鲁棒性。模型在高唤醒和低唤醒情感之间迁移鲁棒;合成基准和感知基准之间的双向迁移表明共享的韵律结构;即使在较小的训练规模下,性能也保持鲁棒。
英文摘要:
Prosodic emphasis varies across languages, emotions, and speaking styles, yet existing emphasis detection models are largely trained and evaluated on monolingual neutral read speech. We introduce MMEE (Multilingual Multi-Emotion Emphasis), a corpus of 10,000 professionally recorded expressive utterances (14.13 hours) across 7 languages and 34 emotion/style categories, with three-level perceptual labels (10 annotations per sample). We benchmark two state-of-the-art architectures under monolingual, cross-lingual, multilingual, cross-emotion, cross-dataset, and data-scale settings. Monolingual models show limited zero-shot transfer, degrading across typologically distant languages, while multilingual training substantially improves robustness. Models transfer robustly between high- and low-arousal emotions; bidirectional transfer between synthetic and perceptual benchmarks suggests shared prosodic structure; and performance stays robust even at smaller training scales.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。