Momentor：通过细粒度时间推理推进视频大型语言模型
Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoning
AI总结：
本文提出Momentor，一种能完成细粒度时间理解任务的Video-LLM，通过自动数据生成引擎构建片段级指令数据集Moment-10M进行训练，在零样本评估中展现出卓越的细粒度时间定位与理解能力。
摘要：
大型语言模型（LLMs）在理解和处理基于文本的任务方面表现出色。许多努力正致力于将这些属性转移到视频模态，这被称为Video-LLMs。然而，现有的Video-LLMs只能捕获粗粒度语义，无法有效处理与特定视频片段的理解或定位相关的任务。鉴于这些挑战，我们提出了Momentor，一种能够完成细粒度时间理解任务的Video-LLM。为了支持Momentor的训练，我们设计了一个自动数据生成引擎，以构建Moment-10M，这是一个包含片段级指令数据的大规模视频指令数据集。我们在Moment-10M上训练Momentor，使其能够执行片段级推理和定位。在多个任务上的零样本评估表明，Momentor在细粒度时间定位的理解和定位方面表现卓越。
英文摘要：
Large Language Models (LLMs) demonstrate remarkable proficiency in comprehending and handling text-based tasks. Many efforts are being made to transfer these attributes to video modality, which are termed Video-LLMs. However, existing Video-LLMs can only capture the coarse-grained semantics and are unable to effectively handle tasks related to comprehension or localization of specific video segments. In light of these challenges, we propose Momentor, a Video-LLM capable of accomplishing fine-grained temporal understanding tasks. To support the training of Momentor, we design an automatic data generation engine to construct Moment-10M, a large-scale video instruction dataset with segment-level instruction data. We train Momentor on Moment-10M, enabling it to perform segment-level reasoning and localization. Zero-shot evaluations on several tasks demonstrate that Momentor excels in fine-grained temporally grounded comprehension and localization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
