GRACE: 基于接地动作中心证据增强视频多模态大语言模型用于观众情感预测
GRACE: Boosting Video MLLMs with Grounded Action-Centric Evidence for Viewer Sentiment Prediction
AI总结：
提出GRACE框架，通过提取时间有序的主谓宾三元组和视觉实体裁剪，增强视频MLLM对细粒度情感线索的提取与推理，在Pitts数据集上提升Qwen2.5-VL和Qwen3-VL性能。
摘要：
视频广告中的观众情感预测旨在推断观众中引发的潜在情感反应。为了弥合展示内容与感受之间的差距，模型必须从显性的视觉叙事、具体的角色-物体交互和可见的文本线索中推断隐藏的观众情感。然而，标准的多模态大语言模型（MLLMs）通常依赖整体帧表示，这使得这些细粒度的情感相关事件隐式化，并复杂化了精确的情感推理。为了解决这个问题，我们提出了一种基于接地动作中心的证据增强框架，通过引入显式事件结构和局部化视觉证据来增强视频MLLMs的线索提取和理解能力。我们的方法从以动作中心的视频描述中提取时间排序的主语-动词-宾语（SVO）三元组和辅助可见文本线索，将主语和宾语实体作为视觉实体裁剪进行接地，然后使MLLM基于这些提取的结构化线索执行线索增强的情感推理。通过这种方式，动作三元组指定“发生了什么”，而接地的视觉实体裁剪将“谁或什么参与每个事件”锚定到具体的视觉证据上。在Pitts数据集上的实验显示，相对于Qwen2.5-VL和Qwen3-VL基线有持续改进。消融研究、在AdsQA上的跨数据集评估以及在情感聚焦的TVQA子集上的迁移实验进一步支持了我们方法的有效性和泛化能力。
英文摘要：
Viewer sentiment prediction in video advertisements aims to infer the latent affective response evoked in the audience. To bridge the gap between what is shown and what is felt, models must deduce hidden viewer emotions from explicit visual narratives, concrete character-object interactions, and visible textual cues. However, standard Multimodal Large Language Models (MLLMs) typically rely on holistic frame representations, which leave these fine-grained, affect-relevant events implicit and complicate precise emotional reasoning. To address this, we propose a grounded action-centric evidence augmentation framework that enhances video MLLMs' clue extraction and comprehension by introducing explicit event structure and localized visual evidence. Our method extracts temporally ordered subject-verb-object (SVO) triplets and auxiliary visible textual cues from action-centric video descriptions, grounds subject and object entities as visual entity crops, and then enables the MLLM to perform clue-enhanced emotional reasoning based on these extracted structured clues. In this way, action triplets specify "what happens", while grounded visual entity crops anchor "who or what participates in each event" to concrete visual evidence. Experiments on the Pitts dataset show consistent improvements over Qwen2.5-VL and Qwen3-VL baselines. Ablation studies, cross-dataset evaluation on AdsQA, and transfer experiments on an emotion-focused TVQA subset further support the effectiveness and generalization of our approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
