VidChain：面向密集视频描述的任务链与基于度量的直接偏好优化
VidChain: Chain-of-Tasks with Metric-based Direct Preference Optimization for Dense Video Captioning
AI总结：
针对视频大语言模型在密集视频描述任务中推理能力未充分发挥、训练目标与评估指标不对齐的问题，提出由任务链和基于度量的直接偏好优化组成的VidChain框架，有效提升细粒度视频理解性能。
摘要：
尽管视频大语言模型（VideoLLMs）在各类任务中取得进展，但它们在细粒度时序理解（如密集视频描述DVC）上仍存在困难。DVC是一项复杂任务，需描述视频内所有事件并进行时序定位，整合了视频分割、视频描述、时序视频 grounding 等多项细粒度任务。以往的VideoLLMs试图单步解决DVC，未能发挥其推理能力。此外，以往VideoLLMs的训练目标未充分反映评估指标，无法提供与目标任务直接对齐的监督。为解决该问题，我们提出名为VidChain的新型框架，由任务链（CoTasks）和基于度量的直接偏好优化（M-DPO）组成。CoTasks将复杂任务分解为一系列子任务，使VideoLLMs能更有效地利用其推理能力。M-DPO使VideoLLM与评估指标对齐，为每个任务提供与指标高度匹配的细粒度监督。将VidChain应用于两种不同的VideoLLMs后，其细粒度视频理解能力持续提升，在两个不同的DVC基准以及时序视频grounding任务上均优于以往的VideoLLMs。代码可在https://github.com/mlvlab/VidChain获取。
英文摘要：
Despite the advancements of Video Large Language Models (VideoLLMs) in various tasks, they struggle with fine-grained temporal understanding, such as Dense Video Captioning (DVC). DVC is a complicated task of describing all events within a video while also temporally localizing them, which integrates multiple fine-grained tasks, including video segmentation, video captioning, and temporal video grounding. Previous VideoLLMs attempt to solve DVC in a single step, failing to utilize their reasoning capability. Moreover, previous training objectives for VideoLLMs do not fully reflect the evaluation metrics, therefore not providing supervision directly aligned to target tasks. To address such a problem, we propose a novel framework named VidChain comprised of Chain-of-Tasks (CoTasks) and Metric-based Direct Preference Optimization (M-DPO). CoTasks decompose a complex task into a sequence of sub-tasks, allowing VideoLLMs to leverage their reasoning capabilities more effectively. M-DPO aligns a VideoLLM with evaluation metrics, providing fine-grained supervision to each task that is well-aligned with metrics. Applied to two different VideoLLMs, VidChain consistently improves their fine-grained video understanding, thereby outperforming previous VideoLLMs on two different DVC benchmarks and also on the temporal video grounding task. Code is available at \url{https://github.com/mlvlab/VidChain}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
