面向视频片段检索的带非对称协同注意力网络的跨模态对比学习
Cross-modal Contrastive Learning with Asymmetric Co-attention Network for Video Moment Retrieval
AI总结：
针对视频文本领域可能存在的信息不对称问题，该研究引入非对称协同注意力网络与动量对比损失，在参数量显著更少的情况下，于TACoS数据集取得SOTA性能，在ActivityNet Captions上取得可比结果。
摘要：
视频片段检索是一项需要视频与文本模态间细粒度交互的挑战性任务。近期图文预训练领域的研究表明，由于视觉序列与文本序列的长度差异，大多数现有预训练模型存在信息不对称问题。我们质疑在需要同时保留空间和时间信息的视频文本领域，是否也存在同样的问题。为此，我们评估了近期提出的一种解决方案：为视频 grounding（视频定位）任务添加非对称协同注意力网络。此外，我们引入动量对比损失，以实现两种模态下鲁棒、有判别力的表征学习。我们发现，集成这些补充模块后，模型在TACoS数据集上的性能优于现有SOTA模型，在ActivityNet Captions数据集上取得了相当的结果，同时参数量相比基线模型显著更少。
英文摘要：
Video moment retrieval is a challenging task requiring fine-grained interactions between video and text modalities. Recent work in image-text pretraining has demonstrated that most existing pretrained models suffer from information asymmetry due to the difference in length between visual and textual sequences. We question whether the same problem also exists in the video-text domain with an auxiliary need to preserve both spatial and temporal information. Thus, we evaluate a recently proposed solution involving the addition of an asymmetric co-attention network for video grounding tasks. Additionally, we incorporate momentum contrastive loss for robust, discriminative representation learning in both modalities. We note that the integration of these supplementary modules yields better performance compared to state-of-the-art models on the TACoS dataset and comparable results on ActivityNet Captions, all while utilizing significantly fewer parameters with respect to baseline.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
