FACE-net：事实校准与情感增强用于检索增强的情感视频描述
FACE-net: Factual Calibration and Emotion Augmentation for Retrieval-enhanced Emotional Video Captioning
AI总结：
本文提出FACE-net框架，通过统一架构协同挖掘事实与情感语义，提供适应性指导以生成准确描述，解决事实-情感偏置问题。
摘要：
情感视频描述（EVC）是一项新兴任务，旨在用视频中表达的内在情感描述事实内容。现有方法感知全局情感线索并结合视频内容生成描述，但生成过程中事实和情感线索挖掘不足及协调问题导致难以处理事实-情感偏置，即不同样本生成时事实和情感需求不同。为此，我们提出一个检索增强框架，通过统一架构协同挖掘事实-情感语义，并提供适应性和准确性的指导，突破了所有样本学习中事实-情感描述的妥协倾向。技术上，我们首先引入外部存储库，通过视频内容检索最相关的句子以增强语义信息。随后，我们的事实校准通过不确定性估计模块将检索信息拆分为主谓宾三元组，并通过视频内容进行自反思和交叉反思以有效挖掘事实语义；同时，我们的渐进视觉情感增强模块利用校准后的事实语义作为专家，与视频内容和情感词典互动生成视觉查询和候选情感，并将它们聚合以适应性地增强每个事实语义的情感。此外，为缓解事实-情感偏置，我们设计了一个动态偏置调整路由模块，以预测并调整样本的偏置程度。
英文摘要：
Emotional Video Captioning (EVC) is an emerging task, which aims to describe factual content with the intrinsic emotions expressed in videos. Existing works perceive global emotional cues and then combine with video content to generate descriptions. However, insufficient factual and emotional cues mining and coordination during generation make their methods difficult to deal with the factual-emotional bias, which refers to the factual and emotional requirements being different in different samples on generation. To this end, we propose a retrieval-enhanced framework with FActual Calibration and Emotion augmentation (FACE-net), which through a unified architecture collaboratively mines factual-emotional semantics and provides adaptive and accurate guidance for generation, breaking through the compromising tendency of factual-emotional descriptions in all sample learning. Technically, we firstly introduces an external repository and retrieves the most relevant sentences with the video content to augment the semantic information. Subsequently, our factual calibration via uncertainty estimation module splits the retrieved information into subject-predicate-object triplets, and self-refines and cross-refines different components through video content to effectively mine the factual semantics; while our progressive visual emotion augmentation module leverages the calibrated factual semantics as experts, interacts with the video content and emotion dictionary to generate visual queries and candidate emotions, and then aggregates them to adaptively augment emotions to each factual semantics. Moreover, to alleviate the factual-emotional bias, we design a dynamic bias adjustment routing module to predict and adjust the degree of bias of a sample.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
