跨语言语音情感识别：人类与自监督模型
Cross-Lingual Speech Emotion Recognition: Humans vs. Self-Supervised Models
AI总结：
本研究对比了人类与自监督模型在跨语言语音情感识别中的表现，通过层级分析和参数高效微调策略，发现模型经知识迁移后可达母语者水平，并揭示了方言对不同情感识别的影响及人机行为差异。
摘要：
利用自监督学习（SSL）模型进行语音情感识别（SER）已被证明是有效的，但探索跨语言场景的研究仍然有限。本研究对人类表现与SSL模型进行了对比分析，首先进行层级分析，并在单语、跨语言和迁移学习背景下探讨参数高效微调策略。我们进一步在话语和片段级别比较了模型与人类的SER能力。此外，我们通过人类评估研究了方言对跨语言SER的影响。我们的发现表明，通过适当的知识迁移，模型能够适应目标语言，并达到与母语者相当的表现。我们还证明了方言对缺乏先前语言和副语言背景的个体的SER具有显著影响。此外，人类和模型在不同情感上表现出不同的行为。这些结果为SSL模型的跨语言SER能力提供了新见解，强调了它们与人类情感感知的相似性和差异性。
英文摘要：
Utilizing Self-Supervised Learning (SSL) models for Speech Emotion Recognition (SER) has proven effective, yet limited research has explored cross-lingual scenarios. This study presents a comparative analysis between human performance and SSL models, beginning with a layer-wise analysis and an exploration of parameter-efficient fine-tuning strategies in monolingual, cross-lingual, and transfer learning contexts. We further compare the SER ability of models and humans at both utterance- and segment-levels. Additionally, we investigate the impact of dialect on cross-lingual SER through human evaluation. Our findings reveal that models, with appropriate knowledge transfer, can adapt to the target language and achieve performance comparable to native speakers. We also demonstrate the significant effect of dialect on SER for individuals without prior linguistic and paralinguistic background. Moreover, both humans and models exhibit distinct behaviors across different emotions. These results offer new insights into the cross-lingual SER capabilities of SSL models, underscoring both their similarities to and differences from human emotion perception.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
