arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.08237cs.SDcs.CLeess.AS
收藏

基于多门机制和神经结构搜索的多语言语音情感识别

Multilingual Speech Emotion Recognition With Multi-Gating Mechanism and Neural Architecture Search

Columbia University(哥伦比亚大学);JP Morgan AI Research(摩根大通人工智能研究院)

Zihan Wang, Qi Meng, HaiFeng Lan, XinRui Zhang, KeHao Guo, Akshat Gupta

摘要:

语音情感识别(SER)将音频分为情感类别,如快乐,愤怒,恐惧,厌恶和中性。虽然语音情感识别(SER)是流行语言的常见应用,但它仍然是低资源语言的问题,即,没有预先训练的语音到文本识别模型的语言。本文首先提出了一种从多个预先训练好的语音模型中提取情感信息的特定语言模型,然后设计了一种多领域模型,可以同时对多种语言进行SER。我们的多域模型采用多门机制,以产生独特的加权特征组合,为每种语言,也搜索特定的神经网络结构,为每种语言通过神经架构搜索模块。此外,我们引入了一个对比辅助损失,以建立更可分离的音频数据表示。我们的实验表明,我们的模型提高了3%的最先进的准确性为德国和法国的14.3%。

英文摘要:

Speech emotion recognition (SER) classifies audio into emotion categories such as Happy, Angry, Fear, Disgust and Neutral. While Speech Emotion Recognition (SER) is a common application for popular languages, it continues to be a problem for low-resourced languages, i.e., languages with no pretrained speech-to-text recognition models. This paper firstly proposes a language-specific model that extract emotional information from multiple pre-trained speech models, and then designs a multi-domain model that simultaneously performs SER for various languages. Our multidomain model employs a multi-gating mechanism to generate unique weighted feature combination for each language, and also searches for specific neural network structure for each language through a neural architecture search module. In addition, we introduce a contrastive auxiliary loss to build more separable representations for audio data. Our experiments show that our model raises the state-of-the-art accuracy by 3% for German and 14.3% for French.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑