面向视频问答的视觉语言模型与符号化基础结合
Bridging Vision Language Models and Symbolic Grounding for Video Question Answering
AI总结：
提出SG-VLM框架，通过提示和视觉定位将冻结的视觉语言模型与符号化场景图结合，在多个基准上改进视频问答的因果与时间推理。
摘要：
视频问答（VQA）要求模型对视频中的空间、时间及因果线索进行推理。近期的视觉语言模型（VLMs）虽取得了强劲结果，但常依赖浅层关联，导致时间基础薄弱且可解释性有限。我们研究将符号化场景图（SGs）作为VQA的中间基础信号。SGs提供结构化的对象-关系表示，以补充VLMs的整体推理。我们提出了SG-VLM，这是一个模块化框架，通过提示和视觉定位将冻结的VLMs与场景图基础相结合。在三个基准（NExT-QA、iVQA、ActivityNet-QA）和多个VLMs（QwenVL、InternVL）上，SG-VLM改进了因果与时间推理并优于先前基线，尽管对强VLMs的提升有限。这些发现突显了符号化基础的潜力与当前局限，为未来视频理解中的混合VLM-符号方法提供了指导。
英文摘要：
Video Question Answering (VQA) requires models to reason over spatial, temporal, and causal cues in videos. Recent vision language models (VLMs) achieve strong results but often rely on shallow correlations, leading to weak temporal grounding and limited interpretability. We study symbolic scene graphs (SGs) as intermediate grounding signals for VQA. SGs provide structured object-relation representations that complement VLMs holistic reasoning. We introduce SG-VLM, a modular framework that integrates frozen VLMs with scene graph grounding via prompting and visual localization. Across three benchmarks (NExT-QA, iVQA, ActivityNet-QA) and multiple VLMs (QwenVL, InternVL), SG-VLM improves causal and temporal reasoning and outperforms prior baselines, though gains over strong VLMs are limited. These findings highlight both the promise and current limitations of symbolic grounding, and offer guidance for future hybrid VLM-symbolic approaches in video understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
