arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.09933cs.SDcs.AIcs.HCcs.LG
收藏

跨数据集泛化 SER 模型需要什么?一项综合基准研究

What Does it Take to Generalize SER Model Across Datasets? A Comprehensive Benchmark

Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);University of British Columbia(不列颠哥伦比亚大学)

Adham Ibrahim, Shady Shehata, Ajinkya Kulkarni, Mukhtar Mohamed, Muhammad Abdul-Mageed

AI总结:

本文针对 SER 跨数据集泛化问题,整合 11 个情感语音数据集构建综合基准,结合过采样、多种评估协议及 Whisper 与说话人无关方法,提升其真实场景泛化能力。

摘要:

语音情感识别(SER)对于增强基于语音的应用中的人机交互至关重要。尽管在特定情感数据集上已有改进,但 SER 在跨真实场景泛化方面仍存在研究空白。本文研究了使 SER 系统跨不同情感数据集实现泛化的方法。具体而言,我们纳入了 11 个情感语音数据集,并展示了一个针对 SER 任务的综合基准。在合并 SER 数据集进行训练时,我们还利用过采样方法应对数据分布不平衡的挑战。此外,我们探索了多种评估协议,以衡量 SER 泛化的熟练程度。在此基础上,我们探究了 Whisper 用于 SER 的潜力,并强调全面评估的重要性。我们的方法旨在通过整合说话人无关方法来推进 SER 技术。

英文摘要:

Speech emotion recognition (SER) is essential for enhancing human-computer interaction in speech-based applications. Despite improvements in specific emotional datasets, there is still a research gap in SER's capability to generalize across real-world situations. In this paper, we investigate approaches to generalize the SER system across different emotion datasets. In particular, we incorporate 11 emotional speech datasets and illustrate a comprehensive benchmark on the SER task. We also address the challenge of imbalanced data distribution using over-sampling methods when combining SER datasets for training. Furthermore, we explore various evaluation protocols for adeptness in the generalization of SER. Building on this, we explore the potential of Whisper for SER, emphasizing the importance of thorough evaluation. Our approach is designed to advance SER technology by integrating speaker-independent methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑