arXiv 2406.071622024-06-11cs.SDcs.AIcs.CLcs.MMeess.AS

EmoBox：多语言多语料库语音情感识别工具包与基准

EmoBox: Multilingual Multi-corpus Speech Emotion Recognition Toolkit and Benchmark

Shanghai Jiao Tong University（上海交通大学）；University of Sheffield（谢菲尔德大学）；Fudan University（复旦大学）

Ziyang Ma, Mingjie Chen, Hezhao Zhang, Zhisheng Zheng, Wenxi Chen, Xiquan Li, Jiaxin Ye, Xie Chen, Thomas Hain

备注：: Accepted by INTERSPEECH 2024. GitHub Repository: https://github.com/emo-box/EmoBox

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出EmoBox多语言多语料库语音情感识别工具包与基准，通过精心设计的数据划分和基于emotion2vec的平衡测试集，在32个数据集、14种语言上评估10个预训练模型，构建了目前规模最大的SER基准。

摘要：

语音情感识别（SER）是人机交互的重要组成部分，受到工业界和学术界的广泛关注。然而，当前SER研究领域长期存在以下问题：1）数据集缺乏合理且通用的划分方式，使得不同模型和方法的比较变得困难。2）没有常用的基准覆盖大量语料库和语言供研究者参考，导致复现成为负担。在本文中，我们提出了EmoBox，一个开箱即用的多语言多语料库语音情感识别工具包，以及用于语料库内和跨语料库设置的基准。对于语料库内设置，我们为不同数据集精心设计了数据划分。对于跨语料库设置，我们采用基础SER模型emotion2vec来缓解标注错误，并获得一个在说话人和情感分布上完全平衡的测试集。基于EmoBox，我们展示了10个预训练语音模型在32个情感数据集、14种语言上的语料库内SER结果，以及在4个数据集上使用完全平衡测试集的跨语料库SER结果。据我们所知，这是跨语言范围和数量规模最大的SER基准。我们希望我们的工具包和基准能够促进社区中SER的研究。

英文摘要：

Speech emotion recognition (SER) is an important part of human-computer interaction, receiving extensive attention from both industry and academia. However, the current research field of SER has long suffered from the following problems: 1) There are few reasonable and universal splits of the datasets, making comparing different models and methods difficult. 2) No commonly used benchmark covers numerous corpus and languages for researchers to refer to, making reproduction a burden. In this paper, we propose EmoBox, an out-of-the-box multilingual multi-corpus speech emotion recognition toolkit, along with a benchmark for both intra-corpus and cross-corpus settings. For intra-corpus settings, we carefully designed the data partitioning for different datasets. For cross-corpus settings, we employ a foundation SER model, emotion2vec, to mitigate annotation errors and obtain a test set that is fully balanced in speakers and emotions distributions. Based on EmoBox, we present the intra-corpus SER results of 10 pre-trained speech models on 32 emotion datasets with 14 languages, and the cross-corpus SER results on 4 datasets with the fully balanced test sets. To the best of our knowledge, this is the largest SER benchmark, across language scopes and quantity scales. We hope that our toolkit and benchmark can facilitate the research of SER in the community.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。