arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.17656cs.CV
收藏

Open-o3-Video-v2:基于可验证时空证据的搜索、定位与推理

Open-o3-Video-v2: Seek, Ground, and Reason with Verifiable Spatio-Temporal Evidence

PKU(北京大学);FDU(复旦大学);NTU(台湾大学);UC Merced(加州大学默塞德分校)

Jiahao Meng, Zhongyuan Bao, Yue Tan, Haiquan Wen, Renrui Zhang, Anna Wang, Jason Li, Yanwei Fu, Yunhai Tong, Ming-Hsuan Yang

AI总结:

Open-o3-Video-v2将视频推理建模为智能体式证据定位过程,构建相关数据集并提出CS-GRPO方法,在V-STaR等基准上较骨干模型实现显著性能提升,验证了主动证据获取与可验证定位的互补性。

摘要:

可靠的视频推理要求模型生成正确答案并提供支撑性视觉证据。Open-o3-Video通过将时间戳和物体级边界框嵌入推理轨迹实现可验证推理,但受限于初始稀疏采样器的帧。我们提出Open-o3-Video-v2,将视频推理建模为统一的智能体式证据定位过程:模型识别缺失信息、获取额外观测、定位支撑推理的时刻与物体,并将该证据写入轨迹后继续推理或作答。为支撑该行为,我们构建了STGR-v2-CoT135K(含5.2万条多轮智能体式定位推理轨迹)和用于强化学习的STGR-v2-RL-13K。我们进一步提出Clue-Seek GRPO(CS-GRPO),该方法追踪工具调用如何改变累积证据,奖励线索覆盖率与逐步增益,并惩罚过度、重复或无信息的检索。在V-STaR基准上,Open-o3-Video-v2-4B取得40.4%的mAM和56.3%的mLGM,分别比其骨干模型Qwen3-VL-4B高出8.5%和12.6%;在通用视频理解、长视频推理和时间定位基准上也实现了持续提升。这些结果表明,主动证据获取与可验证物体级定位对可靠视频推理具有互补作用。代码与模型已在该httpsURL发布。

英文摘要:

Reliable video reasoning requires models to produce correct answers and expose supporting visual evidence. Open-o3-Video enables verifiable reasoning by embedding timestamps and object-level bounding boxes into reasoning traces, but is restricted to frames from an initial sparse sampler. We introduce Open-o3-Video-v2, which formulates video reasoning as a unified agentic evidencegrounded process. The model identifies missing information, acquires additional observations, grounds the moments and objects supporting its reasoning, and writes this evidence into the trajectory before continuing or answering. To support this behavior, we construct STGR-v2-CoT135K, including 52K multi-turn agentic grounded reasoning trajectories, and STGR-v2-RL-13K for reinforcement learning. We further propose Clue-Seek GRPO (CS-GRPO), which tracks how tool calls change accumulated evidence, rewards clue coverage and step-wise gains, and penalizes excessive, repetitive, or uninformative retrieval. On V-STaR, Open-o3-Video-v2-4B achieves 40.4% mAM and 56.3% mLGM, outperforming its Qwen3-VL-4B backbone by 8.5% and 12.6%, respectively. It also delivers consistent gains across general video understanding, long-video reasoning, and temporal grounding benchmarks. These results show that active evidence acquisition and verifiable object-level grounding are complementary for reliable video reasoning. The code and models are released at https://msalab-pku.github.io/projects/Open-o3-Video-v2.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑