面向高效长视频推理的语义与视觉证据：HD-EPIC VQA挑战赛的解决方案
Semantic and Visual Evidence for Efficient Long-Video Reasoning: A Solution for the HD-EPIC VQA Challenge
AI总结：
提出一种统一框架，通过解耦长视频推理为语义证据（粗到细提取全局过程结构）和视觉证据（基于目标的细粒度定位），并采用查询条件证据检索与整合，在HD-EPIC VQA挑战赛中取得竞争性能。
摘要：
理解长格式自我中心视频对于多模态大语言模型（MLLMs）仍然具有挑战性，原因在于有限的上下文长度和对细粒度视觉细节的定位不足。最近提出的HD-EPIC基准突出了这些局限性：即使是强大的长上下文模型，在多样化的视频问答任务中也表现较低。在本文中，我们提出了一个统一框架，将长视频推理解耦为两种互补的证据形式：语义证据和视觉证据。语义证据通过粗到细的提取流程捕获全局过程结构，而基于目标的视觉证据通过边界框和视觉嵌入保留细粒度的定位。在推理过程中，我们将推理形式化为查询条件的证据检索和整合过程，动态地从两个来源选择相关信息。我们的方法在HD-EPIC-VQA挑战赛的多个任务类别中取得了竞争性能。更广泛地说，我们的结果表明，显式地结构化、检索和整合语义与视觉证据对于使用MLLMs进行有效的长视频理解至关重要。
英文摘要：
Understanding long-form egocentric videos remains challenging for multimodal large language models (MLLMs) due to limited context length and insufficient grounding of fine-grained visual details. The recently proposed HD-EPIC benchmark highlights these limitations: even strong long-context models achieve relatively low performance across diverse video question answering tasks. In this paper, we propose a unified framework that decouples long-video reasoning into two complementary forms of evidence: semantic evidence and visual evidence. Semantic evidence captures global procedural structure through a coarse-to-fine extraction pipeline, while object-centric visual evidence preserves fine-grained grounding through bounding boxes and visual embeddings. During inference, we formulate reasoning as a query-conditioned evidence retrieval and integration process, dynamically selecting relevant information from both sources. Our approach achieves competitive performance in the HD-EPIC-VQA Challenge across multiple task categories. More broadly, our results demonstrate that explicitly structuring, retrieving, and integrating semantic and visual evidence is critical for effective long-video understanding with MLLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
