arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1907.03050cs.LGcs.HCeess.ASstat.ML
收藏

联合对齐和预测连续情感注释

Jointly Aligning and Predicting Continuous Emotion Annotations

University of Michigan(密歇根大学);University of Michigan School of Medicine(密歇根大学医学院)

Soheil Khorram, Melvin G McInnis, Emily Mower Provost

摘要:

情绪的时间连续维度描述(例如,唤醒,效价),使研究人员能够表征短期的变化,并捕捉情绪表达的长期趋势。然而,由于由反应时间引起的延迟,连续情感标签通常不与输入语音信号同步,这在人类评估中是固有的。为了应对这一挑战,我们引入了一种新的卷积神经网络(多延迟sinc网络),它能够以端到端的方式同时对齐和预测标签。所提出的网络是卷积层的堆栈,然后是对齐语音信号和情感标签的对齐器网络。该网络使用我们引入的新卷积层(延迟sinc层)来实现。它是一种时移低通(sinc)滤波器,使用基于梯度的算法来学习单个延迟。多个延迟sinc层可以用于补偿作为声学空间的函数的非平稳延迟。我们在两个常见的情感数据集,RECOLA和SEWA上测试了该系统的有效性,并表明这种方法通过学习时变延迟同时预测情感的维度描述符来获得最先进的语音结果。

英文摘要:

Time-continuous dimensional descriptions of emotions (e.g., arousal, valence) allow researchers to characterize short-time changes and to capture long-term trends in emotion expression. However, continuous emotion labels are generally not synchronized with the input speech signal due to delays caused by reaction-time, which is inherent in human evaluations. To deal with this challenge, we introduce a new convolutional neural network (multi-delay sinc network) that is able to simultaneously align and predict labels in an end-to-end manner. The proposed network is a stack of convolutional layers followed by an aligner network that aligns the speech signal and emotion labels. This network is implemented using a new convolutional layer that we introduce, the delayed sinc layer. It is a time-shifted low-pass (sinc) filter that uses a gradient-based algorithm to learn a single delay. Multiple delayed sinc layers can be used to compensate for a non-stationary delay that is a function of the acoustic space. We test the efficacy of this system on two common emotion datasets, RECOLA and SEWA, and show that this approach obtains state-of-the-art speech-only results by learning time-varying delays while predicting dimensional descriptors of emotions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑