解码情绪：面向语音情感识别的语音模型综合多语言研究
Decoding Emotions: A comprehensive Multilingual Study of Speech Models for Speech Emotion Recognition
AI总结：
本文构建涵盖八种语音表示模型、六种语言的 SER 基准，并通过探测实验发现单一最优层特征较使用全部层平均降低 32% 错误率，中间层最关键。
摘要：
基于 transformer 的语音表示模型的最新进展极大地改变了语音处理。然而，在跨多种语言评估这些模型用于语音情感识别（SER）以及考察其内部表示方面，已有研究仍然有限。本文通过提出一个包含八种语音表示模型和六种不同语言的综合 SER 基准来弥补这些空白。我们开展了探测实验，以深入理解这些模型用于 SER 的内部工作机制。我们发现，与使用语音模型所有层特征的系统相比，使用语音模型单一最优层的特征可在七个数据集上平均降低 32% 的错误率。我们还在德语和波斯语上取得了最先进结果。探测结果表明，语音模型的中间层捕获了语音情感识别最重要的情绪信息。
英文摘要：
Recent advancements in transformer-based speech representation models have greatly transformed speech processing. However, there has been limited research conducted on evaluating these models for speech emotion recognition (SER) across multiple languages and examining their internal representations. This article addresses these gaps by presenting a comprehensive benchmark for SER with eight speech representation models and six different languages. We conducted probing experiments to gain insights into inner workings of these models for SER. We find that using features from a single optimal layer of a speech model reduces the error rate by 32\% on average across seven datasets when compared to systems where features from all layers of speech models are used. We also achieve state-of-the-art results for German and Persian languages. Our probing results indicate that the middle layers of speech models capture the most important emotional information for speech emotion recognition.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
