arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2011.09212cs.CL
收藏

基于自监督预训练声学和语言特征的连续语音情感识别

On the use of Self-supervised Pre-trained Acoustic and Linguistic Features for Continuous Speech Emotion Recognition

Allo-Media(阿洛媒体公司)

Manon Macary, Marie Tahon, Yannick Estève, Anthony Rousseau

摘要:

特征提取的预训练是一种越来越多的研究方法,以获得更好的音频和文本内容的连续表示。在目前的工作中,我们使用wav2vec和camemBERT作为自监督学习模型来表示我们的数据,以便在AlloSat(一个描述满意度的大型法国情感数据库)上执行连续的语音情感识别(SER),并在最先进的语料库SEWA上专注于效价,唤醒和喜欢维度。据作者所知,本文提出的第一项研究表明,联合使用wav2vec和BERT类预训练的功能是非常相关的处理连续SER任务,通常具有少量的标记训练数据。通过著名的一致性相关系数(CCC)进行评估,我们的实验表明,当使用MFCC结合word2vec词嵌入在AlloSat数据集上时,我们可以达到0.825而不是0.592的CCC值。

英文摘要:

Pre-training for feature extraction is an increasingly studied approach to get better continuous representations of audio and text content. In the present work, we use wav2vec and camemBERT as self-supervised learned models to represent our data in order to perform continuous emotion recognition from speech (SER) on AlloSat, a large French emotional database describing the satisfaction dimension, and on the state of the art corpus SEWA focusing on valence, arousal and liking dimensions. To the authors' knowledge, this paper presents the first study showing that the joint use of wav2vec and BERT-like pre-trained features is very relevant to deal with continuous SER task, usually characterized by a small amount of labeled training data. Evaluated by the well-known concordance correlation coefficient (CCC), our experiments show that we can reach a CCC value of 0.825 instead of 0.592 when using MFCC in conjunction with word2vec word embedding on the AlloSat dataset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑