VCEMO：面向中文声纹的多模态情感识别
VCEMO: Multi-Modal Emotion Recognition for Chinese Voiceprints
AI总结：
针对中文声纹情感识别缺乏高质量语料库的问题，本文构建了包含日常对话的VCEMO数据集，并提出一种利用共注意力融合语音、文本与外部知识的多模态基准模型，结合对比学习正则化应对数据分布不均，在多项数据集上显著优于SOTA。
摘要：
情感识别能够增强机器对用户命令的人性化响应，而基于声纹的感知系统可以轻松集成到智能手机和音响等常用设备中。尽管拥有最多的使用者，但在使用中文声纹进行情感识别方面，明显缺乏高质量的语料库数据集。因此，本文引入了VCEMO数据集来解决这一缺陷。提出的数据集由日常对话构建，包含超过100名用户和7,747个文本样本。此外，本文提出了一个基于多模态的模型作为基准，该模型利用共注意力结构有效融合了语音、文本和外部知识。该系统采用基于对比学习的正则化来应对数据集分布不均和情感表达多样性的问题。实验表明，所提出的模型在VCEMO和IEMOCAP数据集上相比SOTA有显著提升。代码和数据集将发布用于研究。
英文摘要：
Emotion recognition can enhance humanized machine responses to user commands, while voiceprint-based perception systems can be easily integrated into commonly used devices like smartphones and stereos. Despite having the largest number of speakers, there is a noticeable absence of high-quality corpus datasets for emotion recognition using Chinese voiceprints. Hence, this paper introduces the VCEMO dataset to address this deficiency. The proposed dataset is constructed from everyday conversations and comprises over 100 users and 7,747 textual samples. Furthermore, this paper proposes a multimodal-based model as a benchmark, which effectively fuses speech, text, and external knowledge using a co-attention structure. The system employs contrastive learning-based regulation for the uneven distribution of the dataset and the diversity of emotional expressions. The experiments demonstrate the significant improvement of the proposed model over SOTA on the VCEMO and IEMOCAP datasets. Code and dataset will be released for research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
