让我把话说完：基于整体文本理解的视频时间定位
Let Me Finish My Sentence: Video Temporal Grounding with Holistic Text Understanding
AI总结：
本文针对视频时间定位中忽视查询句整体语义的问题，提出融合整体文本信息的帧级门控机制和跨模态对齐损失，在多个基准上超越现有方法。
摘要：
视频时间定位（VTG）旨在识别视频片段中与文本查询相匹配的视觉帧。近期VTG研究采用交叉注意力机制，将视觉帧和文本查询作为独立的token序列进行关联。然而，这些方法忽略了问题的一个关键方面：对查询句子的整体理解。模型可能捕捉到个别词元与任意视觉帧之间的相关性，却可能遗漏全局含义。为解决这一问题，我们提出两项主要贡献：（1）一种视觉帧级门控机制，融合整体文本信息；（2）跨模态对齐损失，用于学习查询与相关帧之间的细粒度相关性。由此，我们正则化个别词元的影响并抑制不相关的视觉帧。我们证明，该方法在VTG基准上优于最先进的方法，表明整体文本理解能够引导模型关注视频中语义重要的部分。
英文摘要：
Video Temporal Grounding (VTG) aims to identify visual frames in a video clip that match text queries. Recent studies in VTG employ cross-attention to correlate visual frames and text queries as individual token sequences. However, these approaches overlook a crucial aspect of the problem: a holistic understanding of the query sentence. A model may capture correlations between individual word tokens and arbitrary visual frames while possibly missing out on the global meaning. To address this, we introduce two primary contributions: (1) a visual frame-level gate mechanism that incorporates holistic textual information, (2) cross-modal alignment loss to learn the fine-grained correlation between query and relevant frames. As a result, we regularize the effect of individual word tokens and suppress irrelevant visual frames. We demonstrate that our method outperforms state-of-the-art approaches in VTG benchmarks, indicating that holistic text understanding guides the model to focus on the semantically important parts within the video.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
