基于嵌入整合方法的视频问答基础问题生成
Foundational Question Generation for Video Question Answering via an Embedding-Integrated Approach
AI总结：
本文提出FIQ框架，通过生成基础问题增强视频问答模型的推理能力，提升泛化和推理性能。
摘要：
传统视频问答（VQA）方法主要依赖问题-答案（Q&A）对来学习视频内容的时空动态。然而，大多数现有注释都是以事件为中心的，这限制了模型捕捉场景全面背景的能力。缺乏诸如对象类别、空间配置和描述性视觉属性等基本信息，使模型无法形成对环境的完整理解，最终限制了其泛化和推理能力。在本文中，我们介绍了基于嵌入整合方法的视频问答基础问题生成（FIQ），该框架旨在通过提高视频内容的基础理解能力来增强VQA模型的推理能力。FIQ从直接从视频中提取的描述信息中生成Q&A对，从而丰富数据集，增加核心场景级属性。这些生成的对帮助模型更全面地理解视频，从而提高泛化能力和推理性能。此外，我们提出了一种VQ-CAlign模块，将任务特定的问题嵌入与相应的视觉特征对齐，保留关键的上下文线索，并增强对下游任务的适应性。在SUTD-TrafficQA数据集上的实验结果表明，FIQ实现了最先进的性能，超过了现有基线方法。
英文摘要：
Conventional VQA approaches primarily rely on question-answer (Q&A) pairs to learn the spatio-temporal dynamics of video content. However, most existing annotations are event-centric, which restricts the model's ability to capture the comprehensive context of a scene. The lack of fundamental information such as object categories, spatial configurations, and descriptive visual attributes prevents the model from forming a complete understanding of the environment, ultimately limiting its generalization and reasoning capability. In this paper, we introduce Foundational Question Generation for Video Question Answering via an Embedding-Integrated Approach (FIQ), a framework designed to enhance the reasoning capability of VQA models by improving their foundational comprehension of video content. FIQ generates Q&A pairs from descriptive information extracted directly from videos, thereby enriching the dataset with core scene-level attributes. These generated pairs help the model develop a more holistic understanding of the video, leading to improved generalizability and reasoning performance. In addition, we propose a VQ-CAlign module that aligns task-specific question embeddings with corresponding visual features, preserving essential contextual cues and enhancing adaptability to downstream tasks. Experimental results on the SUTD-TrafficQA dataset demonstrate that FIQ achieves state-of-the-art performance, surpassing existing baseline approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
