动画GIF中以人为中心的情感识别
Human-Centered Emotion Recognition in Animated GIFs
摘要:
作为一种直观的情感表达方式,GIF动画图像在社交媒体上得到了广泛的应用。以往的GIF情感识别研究大多未能有效利用GIF的独特属性,这可能会限制识别性能。在这项研究中,我们证明了人类相关的信息在GIF的重要性,并进行以人为中心的GIF情感识别与建议的关键点注意视觉注意力网络(KAVAN)。该框架包括一个面部注意模块和一个分层的片段时间模块。人脸注意模块利用GIF图像内容与人物之间的紧密联系,提取帧级视觉特征,重点关注人脸。然后提出了分层分段LSTM(HS-LSTM)模块,以更好地学习全局GIF表示。我们提出的框架在MIT GIFGIF数据集上的表现优于最先进的。此外,面部注意力模块提供可靠的面部区域掩模预测,这提高了模型的可解释性。
英文摘要:
As an intuitive way of expression emotion, the animated Graphical Interchange Format (GIF) images have been widely used on social media. Most previous studies on automated GIF emotion recognition fail to effectively utilize GIF's unique properties, and this potentially limits the recognition performance. In this study, we demonstrate the importance of human related information in GIFs and conduct human-centered GIF emotion recognition with a proposed Keypoint Attended Visual Attention Network (KAVAN). The framework consists of a facial attention module and a hierarchical segment temporal module. The facial attention module exploits the strong relationship between GIF contents and human characters, and extracts frame-level visual feature with a focus on human faces. The Hierarchical Segment LSTM (HS-LSTM) module is then proposed to better learn global GIF representations. Our proposed framework outperforms the state-of-the-art on the MIT GIFGIF dataset. Furthermore, the facial attention module provides reliable facial region mask predictions, which improves the model's interpretability.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。