arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.06309cs.CVcs.AI
收藏

VideoWeave:一种以数据为中心的高效视频理解方法

VideoWeave: A Data-Centric Approach for Efficient Video Understanding

Stanford University(斯坦福大学);Microsoft Research(微软研究院);University of Wisconsin - Madison(威斯康星大学麦迪逊分校)

Zane Durante, Silky Singh, Arpandeep Khatua, Shobhit Agarwal, Reuben Tan, Yong Jae Lee, Jianfeng Gao, Ehsan Adeli, Li Fei-Fei

AI总结:

VideoWeave通过重新组织训练数据提升视频语言模型的数据效率,无需修改模型架构,实现更高准确率。

摘要:

训练视频-语言模型通常由于处理长帧序列的高成本和标注长视频的有限可用性而变得昂贵。我们提出了VideoWeave,一种简单而有效的改进数据效率的方法,通过构建合成长上下文训练样本将短的、带有描述的视频从现有数据集中拼接在一起。与其修改模型架构或优化目标,VideoWeave重新组织可用的视频-文本对,以在固定计算资源下扩展时间多样性。我们系统地研究了不同数据组成策略,如随机与视觉聚类拼接以及描述增强,对下游视频问答任务性能的影响。在相同的计算约束下,使用VideoWeave训练的模型比传统视频微调模型的准确性更高。我们的结果表明,重新组织训练数据,而不是改变架构,可能为训练视频-语言模型提供了一种简单且可扩展的路径。我们在这里链接了所有实验的代码。

英文摘要:

Training video-language models is often prohibitively expensive due to the high cost of processing long frame sequences and the limited availability of annotated long videos. We present VideoWeave, a simple yet effective approach to improve data efficiency by constructing synthetic long-context training samples that splice together short, captioned videos from existing datasets. Rather than modifying model architectures or optimization objectives, VideoWeave reorganizes available video-text pairs to expand temporal diversity within fixed compute. We systematically study how different data composition strategies like random versus visually clustered splicing and caption enrichment affect downstream performance on downstream video question answering. Under identical compute constraints, models trained with VideoWeave achieve higher accuracy than conventional video finetuning. Our results highlight that reorganizing training data, rather than altering architectures, may offer a simple and scalable path for training video-language models. We link our code for all experiments here.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑