QTG-VQA：面向视频问答系统的问题类型引导架构
QTG-VQA: Question-Type-Guided Architectural for VideoQA Systems
AI总结：
针对视频问答领域问题类型影响研究不足、时间信息表征难等问题，提出融合问题类型引导注意力与自适应学习的QTG-VQA架构，含掩码帧建模技术及适配的评估指标，经实验验证有效。
摘要：
在视频问答（VideoQA）领域，问题类型对VQA系统的影响尽管至关重要，但迄今为止相关研究仍相对不足。然而，问题类型的丰富程度直接决定了模型需要学习的概念范围，进而影响其学习能力的上限。本文聚焦于探究不同问题类型对VQA系统的重要性及其对性能的影响，揭示了由问题类型分布不均导致的学习不充分、模型退化等一系列问题。尤其值得注意的是，不同类型的问题对时间信息的依赖程度存在显著差异，而这类信息的表征恰好是VideoQA相较于ImageQA的核心挑战与难点。为应对这些挑战，我们提出了QTG-VQA——一种融合问题类型引导注意力与自适应学习机制的新型架构。具体而言，针对时间类问题，我们设计了掩码帧建模（Masking Frame Modeling）技术来增强时间建模，旨在促使模型掌握更丰富的视觉-语言关系，并处理更复杂的时间依赖关系。此外，本文还提出了一种适配问题类型的新型评估指标。实验结果证实了我们方法的有效性。
英文摘要：
In the domain of video question answering (VideoQA), the impact of question types on VQA systems, despite its critical importance, has been relatively under-explored to date. However, the richness of question types directly determines the range of concepts a model needs to learn, thereby affecting the upper limit of its learning capability. This paper focuses on exploring the significance of different question types for VQA systems and their impact on performance, revealing a series of issues such as insufficient learning and model degradation due to uneven distribution of question types. Particularly, considering the significant variation in dependency on temporal information across different question types, and given that the representation of such information coincidentally represents a principal challenge and difficulty for VideoQA as opposed to ImageQA. To address these challenges, we propose QTG-VQA, a novel architecture that incorporates question-type-guided attention and adaptive learning mechanism. Specifically, as to temporal-type questions, we design Masking Frame Modeling technique to enhance temporal modeling, aimed at encouraging the model to grasp richer visual-language relationships and manage more intricate temporal dependencies. Furthermore, a novel evaluation metric tailored to question types is introduced. Experimental results confirm the effectiveness of our approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
