arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2002.12766eess.AScs.SD
收藏

使用深度3D特征和序列建模在野外进行多模态连续效价和唤醒预测

Multi-Modal Continuous Valence And Arousal Prediction in the Wild Using Deep 3D Features and Sequence Modeling

Accenture Technology Labs, Bangalore, India(埃森哲科技实验室(班加罗尔))

Sowmya Rasipuram, Junaid Hamid Bhat, Anutosh Maitra

摘要:

在野外连续的影响预测是一个非常有趣的问题,是具有挑战性的连续预测涉及大量的计算。本文介绍了在我们的贡献中使用的方法和技术来预测连续的情感维度,即,Aff-Wild 2数据库中ABAW竞争的效价和唤醒。Aff-Wild 2数据库由在帧水平上标记为效价和唤醒的野生视频组成。我们提出的方法使用融合的音频和视频特征(多模态)提取使用国家的最先进的方法。这些音频-视频特征用于训练基于门控递归单元(GRU)的序列到序列模型。我们展示了有希望的结果验证数据与简单的架构。该方法的总体效价和唤醒度分别为0.22和0.34,优于竞争基线0.14和0.24。

英文摘要:

Continuous affect prediction in the wild is a very interesting problem and is challenging as continuous prediction involves heavy computation. This paper presents the methodologies and techniques used in our contribution to predict continuous emotion dimensions i.e., valence and arousal in ABAW competition on Aff-Wild2 database. Aff-Wild2 database consists of videos in the wild labelled for valence and arousal at frame level. Our proposed methodology uses fusion of both audio and video features (multi-modal) extracted using state-of-the-art methods. These audio-video features are used to train a sequence-to-sequence model that is based on Gated Recurrent Units (GRU). We show promising results on validation data with simple architecture. The overall valence and arousal of the proposed approach is 0.22 and 0.34, which is better than the competition baseline of 0.14 and 0.24 respectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑