arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2605.13803cs.CV

EvoGround:用于视频时间定位的自进化视频智能体

EvoGround: Self-Evolving Video Agents for Video Temporal Grounding

Seoul National University(首尔国立大学);Northeastern University(东北大学)

Minjoon Jung, Byoung-Tak Zhang, Lorenzo Torresani

AI总结:

本文提出EvoGround框架,通过两个自进化智能体自学习视频时间定位,无需人工标注数据,在多个基准上表现优异。

摘要:

视频时间定位(VTG)接收一个未剪辑的视频和一个自然语言查询作为输入,并定位最符合查询的时间时刻。现有方法依赖于大规模、任务特定的数据集,需要昂贵的手动标注。我们引入了EvoGround,一个包含两个耦合自进化智能体的框架,一个提议者和一个求解者,从原始视频中学习时间定位,而无需任何人工标注数据。提议者从原始视频中生成查询-时刻对,而求解者学习将它们定位并反馈信号以改善提议者。通过这种自我强化的强化学习循环,两个智能体从相同的主干网络初始化,并在迭代中相互改进。在2.5K未标注视频上训练后,EvoGround在多个VTG基准测试中匹配或超越了完全监督的模型,同时在没有人工标注的情况下成为最先进的细粒度视频描述生成器。

英文摘要:

Video temporal grounding (VTG) takes an untrimmed video and a natural-language query as input and localizes the temporal moment that best matches the query. Existing methods rely on large, task-specific datasets requiring costly manual annotation. We introduce EvoGround, a framework of two coupled self-evolving agents, a proposer and a solver, that learn temporal grounding from raw videos without any human-labeled data. The proposer generates query--moment pairs from raw videos, while the solver learns to ground them and feeds back signals that improve the proposer in return. Through this self-reinforcing reinforcement-learning loop, the two agents are initialized from the same backbone and mutually improve across iterations. Trained on 2.5K unlabeled videos, EvoGround matches or surpasses fully supervised models across multiple VTG benchmarks, while emerging as a state-of-the-art fine-grained video captioner without manual labels.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑