基于可学习图和图初始网络的动态情感建模
Dynamic Emotion Modeling with Learnable Graphs and Graph Inception Network
摘要：
人类情感是使用各种动态数据形式来表达、感知和捕捉的，例如语音（口头）、视频（面部表情）和运动传感器（身体姿势）。我们提出了一种通用的方法来情感识别，可以适应跨模态的动态数据建模为结构化图。图方法背后的动机是在不影响性能的情况下构建紧凑的模型。为了缓解最佳图构造的问题，我们将其作为联合图学习和分类任务。为此，我们提出了可学习的图形初始网络（L-GrIN），共同学习识别情感和识别动态数据中的底层图形结构。我们的架构包括多个新颖的组件：一个新的图卷积操作，一个图的初始层，可学习的邻接，和一个可学习的池函数，产生一个图级嵌入。我们评估了五个基准的情绪识别数据库，跨越三种不同的方式（视频，音频，动作捕捉），其中每个数据库捕捉以下情绪线索之一：面部表情，语音和身体姿态的架构。我们在所有五个数据库上都实现了最先进的性能，优于几个竞争基线和相关的现有方法。我们的图形架构显示出卓越的性能，参数显著减少（与卷积或递归神经网络相比），有望适用于资源受限的设备。
英文摘要：
Human emotion is expressed, perceived and captured using a variety of dynamic data modalities, such as speech (verbal), videos (facial expressions) and motion sensors (body gestures). We propose a generalized approach to emotion recognition that can adapt across modalities by modeling dynamic data as structured graphs. The motivation behind the graph approach is to build compact models without compromising on performance. To alleviate the problem of optimal graph construction, we cast this as a joint graph learning and classification task. To this end, we present the Learnable Graph Inception Network (L-GrIN) that jointly learns to recognize emotion and to identify the underlying graph structure in the dynamic data. Our architecture comprises multiple novel components: a new graph convolution operation, a graph inception layer, learnable adjacency, and a learnable pooling function that yields a graph-level embedding. We evaluate the proposed architecture on five benchmark emotion recognition databases spanning three different modalities (video, audio, motion capture), where each database captures one of the following emotional cues: facial expressions, speech and body gestures. We achieve state-of-the-art performance on all five databases outperforming several competitive baselines and relevant existing methods. Our graph architecture shows superior performance with significantly fewer parameters (compared to convolutional or recurrent neural networks) promising its applicability to resource-constrained devices.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
