利用辅助字幕进行视频定位
Exploiting Auxiliary Caption for Video Grounding
AI总结：
本文提出辅助字幕网络（ACNet），通过引入密集视频字幕生成、非辅助字幕抑制、字幕引导注意力（CGA）和非对称跨模态对比学习（ACCL），有效利用辅助字幕增强视频定位性能，在三个公开数据集上显著优于现有方法。
摘要：
视频定位旨在从未剪辑的视频中定位与给定查询句子匹配的感兴趣片段。以往的工作忽略了视频标注中的稀疏性困境，这导致无法提供数据集中潜在事件与查询句子之间的上下文信息。在本文中，我们认为利用描述一般动作的易得字幕，即本文定义的辅助字幕，将显著提升性能。为此，我们提出了一种用于视频定位的辅助字幕网络（ACNet）。具体而言，我们首先引入密集视频字幕生成方法来生成密集字幕，然后通过非辅助字幕抑制（NACS）获得辅助字幕。为了捕捉辅助字幕中的潜在信息，我们提出了字幕引导注意力（CGA），将辅助字幕与查询句子之间的语义关系投影到时间空间中，并将其融合到视觉表示中。考虑到辅助字幕与真实值之间的差距，我们提出了非对称跨模态对比学习（ACCL），以构建更多的负样本对，从而最大化跨模态互信息。在三个公开数据集（即ActivityNet Captions、TACoS和ActivityNet-CG）上的大量实验表明，我们的方法显著优于现有最先进的方法。
英文摘要：
Video grounding aims to locate a moment of interest matching the given query sentence from an untrimmed video. Previous works ignore the {sparsity dilemma} in video annotations, which fails to provide the context information between potential events and query sentences in the dataset. In this paper, we contend that exploiting easily available captions which describe general actions, i.e., auxiliary captions defined in our paper, will significantly boost the performance. To this end, we propose an Auxiliary Caption Network (ACNet) for video grounding. Specifically, we first introduce dense video captioning to generate dense captions and then obtain auxiliary captions by Non-Auxiliary Caption Suppression (NACS). To capture the potential information in auxiliary captions, we propose Caption Guided Attention (CGA) project the semantic relations between auxiliary captions and query sentences into temporal space and fuse them into visual representations. Considering the gap between auxiliary captions and ground truth, we propose Asymmetric Cross-modal Contrastive Learning (ACCL) for constructing more negative pairs to maximize cross-modal mutual information. Extensive experiments on three public datasets (i.e., ActivityNet Captions, TACoS and ActivityNet-CG) demonstrate that our method significantly outperforms state-of-the-art methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
