ChatVTG：通过与视频对话大语言模型聊天进行视频时序定位
ChatVTG: Video Temporal Grounding via Chat with Video Dialogue Large Language Models
AI总结：
提出ChatVTG方法，利用视频对话大语言模型生成多粒度描述并匹配查询，实现零样本视频时序定位，通过时刻细化提升精度，在多个数据集上超越现有零样本方法。
摘要：
视频时序定位（VTG）旨在从未剪辑视频中定位与给定自然语言查询相对应的特定片段。现有VTG方法主要依赖监督学习和大量标注数据，这不仅耗费人力且易受人类偏见影响。为应对这些挑战，我们提出了ChatVTG，一种利用Video Dialogue Large Language Models（视频对话大语言模型）实现零样本视频时序定位的新方法。ChatVTG借助视频对话LLMs生成多粒度片段描述，并将这些描述与给定查询匹配以实现粗粒度时序定位，从而避免了对配对标注数据的需求。此外，为获得更精确的时序定位结果，我们采用时刻细化处理细粒度描述提案。在包括Charades-STA、ActivityNet-Captions和TACoS三个主流VTG数据集上的大量实验证明了ChatVTG的有效性。ChatVTG超越了当前零样本方法的性能。
英文摘要：
Video Temporal Grounding (VTG) aims to ground specific segments within an untrimmed video corresponding to the given natural language query. Existing VTG methods largely depend on supervised learning and extensive annotated data, which is labor-intensive and prone to human biases. To address these challenges, we present ChatVTG, a novel approach that utilizes Video Dialogue Large Language Models (LLMs) for zero-shot video temporal grounding. Our ChatVTG leverages Video Dialogue LLMs to generate multi-granularity segment captions and matches these captions with the given query for coarse temporal grounding, circumventing the need for paired annotation data. Furthermore, to obtain more precise temporal grounding results, we employ moment refinement for fine-grained caption proposals. Extensive experiments on three mainstream VTG datasets, including Charades-STA, ActivityNet-Captions, and TACoS, demonstrate the effectiveness of ChatVTG. Our ChatVTG surpasses the performance of current zero-shot methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
