arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2602.20159cs.CVcs.AIcs.LGcs.MMcs.RO
收藏

一个非常大的视频推理套件

A Very Big Video Reasoning Suite

University of California, Berkeley(加州大学伯克利分校);Nanyang Technological University(南洋理工大学);Northeastern University(东北大学);University of Tübingen(图宾根大学);Johns Hopkins University(约翰霍普金斯大学);University of Michigan(密歇根大学);University of Southern California(南加州大学);Washington University in St. Louis(圣路易斯华盛顿大学);Shanghai Jiao Tong University(上海交通大学);East China Normal University(华东师范大学);Stanford University(斯坦福大学);University of Texas at Austin(得克萨斯大学奥斯汀分校);University of California, Los Angeles(加州大学洛杉矶分校);Cornell University(康奈尔大学);San Jose State University(圣何塞州立大学);University of California, Irvine(加州大学尔湾分校);Technical University of Munich(慕尼黑技术大学);University of California, San Diego(加州大学圣地亚哥分校);Imperial College London(伦敦帝国学院);University of Wisconsin--Madison(威斯康星大学麦迪逊分校);University of Edinburgh(爱丁堡大学);Hong Kong University of Science(香港科学大学);New York University(纽约大学);Auburn University(阿伯丁大学);Columbia University(哥伦比亚大学);University of Bristol(布里斯托大学);University of Waterloo(滑铁卢大学);The Chinese University of Hong Kong(香港中文大学);Carnegie Mellon University(卡内基梅隆大学);University of Oxford(牛津大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校)
Maijunxian Wang, Ruisi Wang, Juyi Lin, Ran Ji, Thaddäus Wiedemer, Qingying Gao, Dezhi Luo, Yaoyao Qian, Lianyu Huang, Zelong Hong, Jiahui Ge, Qianli Ma, Hang He… 展开作者

Maijunxian Wang, Ruisi Wang, Juyi Lin, Ran Ji, Thaddäus Wiedemer, Qingying Gao, Dezhi Luo, Yaoyao Qian, Lianyu Huang, Zelong Hong, Jiahui Ge, Qianli Ma, Hang He, Yifan Zhou, Lingzi Guo, Lantao Mei, Jiachen Li, Hanwen Xing, Tianqi Zhao, Fengyuan Yu, Weihang Xiao, Yizheng Jiao, Jianheng Hou, Danyang Zhang, Pengcheng Xu, Boyang Zhong, Zehong Zhao, Gaoyun Fang, John Kitaoka, Yile Xu, Hua Xu, Kenton Blacutt, Tin Nguyen, Siyuan Song, Haoran Sun, Shaoyue Wen, Linyang He, Runming Wang, Yanzhi Wang, Mengyue Yang, Ziqiao Ma, Raphaël Millière, Freda Shi, Nuno Vasconcelos, Daniel Khashabi, Alan Yuille, Yilun Du, Ziming Liu, Bo Li, Dahua Lin, Ziwei Liu, Vikash Kumar, Yijiang Li, Lei Yang, Zhongang Cai, Hokin Deng

AI总结:

VBVR数据集和评估框架旨在解决视频推理能力研究中的大规模数据缺乏问题,通过大规模实验观察到对未见任务的泛化能力。

摘要:

视频模型的快速发展主要集中在视觉质量上,其推理能力却鲜有研究。视频推理基于时空一致的视觉环境,这种环境超越了文本自然捕捉的能力,能够直观地对时空结构进行推理,如连续性、交互性和因果性。然而,系统研究视频推理及其扩展行为受到缺乏大规模训练数据的阻碍。为了解决这一差距,我们引入了非常大的视频推理(VBVR)数据集,这是一个前所未有的大规模资源,涵盖200个经过精心挑选的推理任务,遵循原则性的分类学,并包含超过一百万段视频片段,规模大约比现有数据集大三个数量级。我们进一步提出了VBVR-Bench,一个可验证的评估框架,它超越了基于模型的评判,通过整合基于规则的人类对齐评分者,使视频推理能力的可重复和可解释的诊断成为可能。利用VBVR套件,我们进行了首次大规模的视频推理扩展研究,并观察到对未见过的推理任务的新兴泛化迹象。共同地,VBVR为可推广的视频推理下一阶段的研究奠定了基础。数据、基准工具包和模型可在https://video-reason.com/ 公开获取。

英文摘要:

Rapid progress in video models has largely focused on visual quality, leaving their reasoning capabilities underexplored. Video reasoning grounds intelligence in spatiotemporally consistent visual environments that go beyond what text can naturally capture, enabling intuitive reasoning over spatiotemporal structure such as continuity, interaction, and causality. However, systematically studying video reasoning and its scaling behavior is hindered by the lack of large-scale training data. To address this gap, we introduce the Very Big Video Reasoning (VBVR) Dataset, an unprecedentedly large-scale resource spanning 200 curated reasoning tasks following a principled taxonomy and over one million video clips, approximately three orders of magnitude larger than existing datasets. We further present VBVR-Bench, a verifiable evaluation framework that moves beyond model-based judging by incorporating rule-based, human-aligned scorers, enabling reproducible and interpretable diagnosis of video reasoning capabilities. Leveraging the VBVR suite, we conduct one of the first large-scale scaling studies of video reasoning and observe early signs of emergent generalization to unseen reasoning tasks. Together, VBVR lays a foundation for the next stage of research in generalizable video reasoning. The data, benchmark toolkit, and models are publicly available at https://video-reason.com/?v=vbvr .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑