面向细粒度语音情感识别的情感神经转导器
Emotion Neural Transducer for Fine-Grained Speech Emotion Recognition
AI总结：
本文提出情感神经转导器（ENT），通过ASR联合训练、情感格与格最大池化及blank情感指示实现细粒度语音情感识别，在IEMOCAP和ZED上优于现有方法。
摘要：
语音情感识别（SER）的主流范式是识别整个话语的单一情感标签。这类工作忽视了细时间粒度下的情感动态，并且大多未能显式利用语音信号中的语言信息。在本文中，我们提出了用于细粒度语音情感识别的情感神经转导器，并与自动语音识别（ASR）进行联合训练。我们首先通过情感联合网络扩展典型的神经转导器，构建用于细粒度SER的情感格。然后，我们在对齐格上提出格最大池化，以促进区分情感帧和非情感帧。为了使细粒度SER适应转导器推理方式，我们进一步让ASR的特殊符号blank同时作为潜在的情感指示符，从而得到因子化情感神经转导器。对于典型的话语级SER，我们的ENT模型在低词错误率条件下于IEMOCAP上优于最先进方法。在IEMOCAP和最新的语音情感分离数据集ZED上的实验也证明了细粒度情感建模的优越性。我们的代码可在https://github.com/ECNU-Cross-Innovation-Lab/ENT获取。
英文摘要：
The mainstream paradigm of speech emotion recognition (SER) is identifying the single emotion label of the entire utterance. This line of works neglect the emotion dynamics at fine temporal granularity and mostly fail to leverage linguistic information of speech signal explicitly. In this paper, we propose Emotion Neural Transducer for fine-grained speech emotion recognition with automatic speech recognition (ASR) joint training. We first extend typical neural transducer with emotion joint network to construct emotion lattice for fine-grained SER. Then we propose lattice max pooling on the alignment lattice to facilitate distinguishing emotional and non-emotional frames. To adapt fine-grained SER to transducer inference manner, we further make blank, the special symbol of ASR, serve as underlying emotion indicator as well, yielding Factorized Emotion Neural Transducer. For typical utterance-level SER, our ENT models outperform state-of-the-art methods on IEMOCAP in low word error rate. Experiments on IEMOCAP and the latest speech emotion diarization dataset ZED also demonstrate the superiority of fine-grained emotion modeling. Our code is available at https://github.com/ECNU-Cross-Innovation-Lab/ENT.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
