arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2312.11974cs.SDcs.HCeess.AS
收藏

Ms-senet:通过挤压-激励模块的多尺度特征融合增强语音情感识别

Ms-senet: Enhancing Speech Emotion Recognition Through Multi-scale Feature Fusion With Squeeze-and-excitation Blocks

Qifu Technology(奇富科技);School of Computer Science, Fudan University(复旦大学计算机科学学院)

Mengbo Li, Yuanzhong Zheng, Dichucheng Li, Yulun Wu, Yaoxuan Wang, Haojun Fei

AI总结:

针对语音情感识别中时空特征学习不足的问题,本文提出Ms-senet方法,利用多尺度CNN与SE模块融合特征,并通过跳跃连接和空间丢弃防止过拟合,在六个基准数据集上平均UAR和WAR分别提升1.62%和1.32%。

摘要:

语音情感识别(SER)已成为人机交互中日益增长的研究焦点。时空特征在SER中起着至关重要的作用,然而当前研究缺乏全面的时空特征学习。本文旨在通过提出一种新方法来解决这一不足。在本文中,我们采用具有不同卷积核大小的卷积神经网络(CNN)进行空间和时间特征提取。此外,我们引入挤压-激励(SE)模块来捕获并融合多尺度特征,促进有效的信息融合,以改进情感识别并加深对语音情感时间演化的理解。此外,我们采用跳跃连接和空间丢弃(SD)层来防止过拟合并增加模型的深度。我们的方法优于先前的最先进方法,在六个基准SER数据集上平均UAR和WAR分别提高了1.62%和1.32%。进一步的实验表明,我们的方法在低资源条件下能够充分提取时空特征。

英文摘要:

Speech Emotion Recognition (SER) has become a growing focus of research in human-computer interaction. Spatiotemporal features play a crucial role in SER, yet current research lacks comprehensive spatiotemporal feature learning. This paper focuses on addressing this gap by proposing a novel approach. In this paper, we employ Convolutional Neural Network (CNN) with varying kernel sizes for spatial and temporal feature extraction. Additionally, we introduce Squeeze-and-Excitation (SE) modules to capture and fuse multi-scale features, facilitating effective information fusion for improved emotion recognition and a deeper understanding of the temporal evolution of speech emotion. Moreover, we employ skip connections and Spatial Dropout (SD) layers to prevent overfitting and increase the model's depth. Our method outperforms the previous state-of-the-art method, achieving an average UAR and WAR improvement of 1.62% and 1.32%, respectively, across six benchmark SER datasets. Further experiments demonstrated that our method can fully extract spatiotemporal features in low-resource conditions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑