arXiv 2604.252762026-04-28cs.CV

OmniVTG：一种大规模数据集和开放世界视频时间定位的训练范式

OmniVTG: A Large-Scale Dataset and Training Paradigm for Open-World Video Temporal Grounding

Wangxuan Institute of Computer Technology, Peking University（北京大学王轩计算机技术研究所）；State Key Laboratory of General Artificial Intelligence, Peking University（北京大学通用人工智能国家重点实验室）；Central Media Technology Institute, Huawei Technologies Ltd.（华为技术有限公司中央媒体技术研究所）；PKU-WUHAN Institute for Artificial Intelligence, Peking University（北京大学武汉人工智能研究所）

Minghang Zheng, Zihao Yin, Yi Yang, Yuxin Peng, Yang Liu

备注：: CVPR 2026

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出OmniVTG数据集和Self-Correction Chain-of-Thought训练范式，通过语义覆盖迭代扩展管道构建大规模数据集，并利用多模态大语言模型的密集描述能力提升视频时间定位性能。

摘要：

视频时间定位（VTG）任务在开放世界场景中面临数据规模有限和语义多样性不足的挑战，导致常见与罕见概念间性能差距。本文引入OmniVTG，通过语义覆盖迭代扩展管道构建大规模数据集，并提出Self-Correction Chain-of-Thought（CoT）训练范式，利用多模态大语言模型（MLLMs）的密集描述能力提升视频时间定位能力。通过三个阶段的训练流程（SFT、CoT微调、强化学习）增强模型的自我修正能力。实验表明，该方法在OmniVTG数据集上表现优异，并在四个现有VTG基准测试中取得最先进的零样本性能。

英文摘要：

Video Temporal Grounding (VTG), the task of localizing video segments from text queries, struggles in open-world settings due to limited dataset scale and semantic diversity, causing performance gaps between common and rare concepts. To overcome these limitations, we introduce OmniVTG, a new large-scale dataset for open-world VTG, coupled with a Self-Correction Chain-of-Thought (CoT) training paradigm designed to enhance the grounding capabilities of Multimodal Large Language Models (MLLMs). Our OmniVTG is constructed via a novel Semantic Coverage Iterative Expansion pipeline, which first identifies gaps in the vocabulary of existing datasets and collects videos that are highly likely to contain these target concepts. For high-quality annotation, we leverage the insight that modern MLLMs excel at dense captioning more than direct grounding and design a caption-centric data engine to prompt MLLMs to generate dense, timestamped descriptions. Beyond the dataset, we observe that simple supervised finetuning (SFT) is insufficient, as a performance gap between rare and common concepts still persists. We find that MLLMs' video understanding ability significantly surpasses their direct grounding ability. Based on this, we propose a Self-Correction Chain-of-Thought (CoT) training paradigm. We train the MLLM to first predict, then use its understanding capabilities to reflect on and refine its own predictions. This capability is instilled via a three-stage pipeline of SFT, CoT finetuning, and reinforcement learning. Extensive experiments show our approach not only excels at open-world grounding in our OmniVTG dataset but also achieves state-of-the-art zero-shot performance on four existing VTG benchmarks. Code is available at https://github.com/oceanflowlab/OmniVTG.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。