Affect2MM：基于情感因果关系的多媒体内容情感分析
Affect2MM: Affective Analysis of Multimedia Content Using Emotion Causality
摘要：
我们提出了Affect 2 MM，这是一种用于多媒体内容时间序列情感预测的学习方法。我们的目标是自动捕捉角色在现实生活中以人为中心的情况和行为中所描绘的各种情绪。我们使用情绪因果关系理论的思想来计算模型，并确定电影片段中引发的情绪状态。Affect 2 MM使用基于注意力的方法和Granger因果关系明确地对时间因果关系进行建模。我们使用各种组件，如演员的面部特征，场景理解，视觉美学，动作/情况描述，和电影剧本，以获得情感丰富的表示来理解和感知场景。我们使用基于LSTM的学习模型进行情绪感知。为了评估我们的方法，我们分析和比较了我们在三个数据集上的性能，SENDv 1，MovieGraphs和LIRIS-ACCEDE数据集，并观察到所有三个数据集的SOTA方法的性能平均提高了10-15%。
英文摘要：
We present Affect2MM, a learning method for time-series emotion prediction for multimedia content. Our goal is to automatically capture the varying emotions depicted by characters in real-life human-centric situations and behaviors. We use the ideas from emotion causation theories to computationally model and determine the emotional state evoked in clips of movies. Affect2MM explicitly models the temporal causality using attention-based methods and Granger causality. We use a variety of components like facial features of actors involved, scene understanding, visual aesthetics, action/situation description, and movie script to obtain an affective-rich representation to understand and perceive the scene. We use an LSTM-based learning model for emotion perception. To evaluate our method, we analyze and compare our performance on three datasets, SENDv1, MovieGraphs, and the LIRIS-ACCEDE dataset, and observe an average of 10-15% increase in the performance over SOTA methods for all three datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
