Number it：像翻阅漫画一样对视频进行时间定位
Number it: Temporal Grounding Videos like Flipping Manga
AI总结：
针对视频大语言模型在视频时间定位任务上的能力短板，提出为每帧添加唯一数字标识的NumPro方法，无需额外成本即可提升模型性能，微调后刷新该任务最优指标。
摘要：
视频大语言模型（Vid-LLM）在面向问答对话的视频内容理解方面已取得显著进展，但它们难以将这种视觉理解能力拓展至需要精确定位时间位置的任务，即视频时间定位（VTG）。为填补这一空白，我们提出了Number-Prompt（NumPro，数字提示）这一新颖方法，通过为每个视频帧添加唯一数字标识符，助力Vid-LLM搭建起视觉理解与时间定位之间的桥梁。NumPro将视频视为带编号的帧图像序列，把VTG转化为一个直观的过程：按顺序翻阅漫画分镜。这使得Vid-LLM能够“读取”事件时间线，将视觉内容与对应的时间信息准确关联起来。我们的实验表明，NumPro在不增加额外计算成本的情况下，显著提升了顶尖Vid-LLM的VTG性能。此外，在NumPro增强的数据集上进行微调后，该方法刷新了VTG的当前最优水平，在时刻检索任务的mIoU指标上较此前最优方法最高提升6.9%，在高光检测任务的mAP指标上最高提升8.5%。代码将发布于https://github.com/yongliang-wu/NumPro。
英文摘要：
Video Large Language Models (Vid-LLMs) have made remarkable advancements in comprehending video content for QA dialogue. However, they struggle to extend this visual understanding to tasks requiring precise temporal localization, known as Video Temporal Grounding (VTG). To address this gap, we introduce Number-Prompt (NumPro), a novel method that empowers Vid-LLMs to bridge visual comprehension with temporal grounding by adding unique numerical identifiers to each video frame. Treating a video as a sequence of numbered frame images, NumPro transforms VTG into an intuitive process: flipping through manga panels in sequence. This allows Vid-LLMs to "read" event timelines, accurately linking visual content with corresponding temporal information. Our experiments demonstrate that NumPro significantly boosts VTG performance of top-tier Vid-LLMs without additional computational cost. Furthermore, fine-tuning on a NumPro-enhanced dataset defines a new state-of-the-art for VTG, surpassing previous top-performing methods by up to 6.9\% in mIoU for moment retrieval and 8.5\% in mAP for highlight detection. The code will be available at https://github.com/yongliang-wu/NumPro.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
