先路由后查看：面向长视频理解的查询自适应证据获取
Routing Before Looking: Query-Adaptive Evidence Acquisition for Long-form Video Understanding
- Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
- Tencent（腾讯）
- Wuhan University（武汉大学）
- Southeast University（东南大学）
- National University of Singapore（新加坡国立大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对长视频理解中查询需求与证据获取策略不匹配的问题，提出Route2Look框架，通过查询自适应路由策略选择证据获取工具，实现最优性能与帧效率。
AI中文摘要：
对于视频智能体而言，长视频理解仍具挑战性，原因在于查询需求与证据获取策略不匹配。尽管近期的感知前规划方法优于查询无关的流程，但它们通常依赖单一主导策略，要么是基于生成的策略，要么是基于检索的策略，这限制了其处理多样查询需求的能力。我们提出Route2Look，这是一个面向长视频理解的查询自适应证据获取的轻量级、模型无关框架。Route2Look在“路由-查看-记忆”循环中运行，包含三个工具：用于整体上下文的全局浏览、用于显式时间线索的时间定位、用于语义搜索的语义检索。核心组件是路由策略，它根据查询动态选择证据获取工具。为构建该策略，Route2Look采用两阶段设计：首先通过基于生成的轨迹与基于检索的轨迹之间的差异对比分析提炼路由技能，然后在推理阶段应用提炼的技能，结合硬路由规则和“继续或停止”标准。在具有挑战性的长视频基准上的实验表明，Route2Look在保持跨数据集和查询类型的强帧效率的同时，实现了最先进的性能。Oracle路由分析进一步揭示了查询自适应证据获取对未来长视频理解的潜力。
英文摘要：
Long-form video understanding remains challenging for video agents due to the mismatch between query demands and evidence acquisition strategies. Although recent planning-before-perception methods outperform query-agnostic pipelines, they often rely on a single dominant strategy, either generation-based strategy or retrieval-based strategy, limiting their ability to handle diverse query demands. We propose Route2Look, a lightweight and model-agnostic framework for query-adaptive evidence acquisition in long-form video understanding. Route2Look operates in a Route-Look-Memorize loop with three tools: Global Browse for holistic context, Temporal Ground for explicit temporal cues, and Semantic Retrieve for semantic search. The core component is a routing policy that dynamically selects evidence acquisition tools based on the query. To build this policy, Route2Look adopts a two-stage design: first distilling the routing skill from differential contrastive analysis between generation-based and retrieval-based trajectories, and then applying the distilled skill with hard routing rules and continue-or-stop criteria during inference. Experiments on challenging long-video benchmarks show that Route2Look achieves state-of-the-art performance while maintaining strong frame efficiency across datasets and query types. Oracle routing analysis further reveals the potential of query-adaptive evidence acquisition for future long-form video understanding.
