利用时间全局文本知识的视频句子定位
Video sentence grounding with temporally global textual knowledge
AI总结：
针对视频句子定位中模态间的领域差距问题，提出伪查询中介网络（PIN），利用时间全局文本知识的伪查询特征，通过对比学习和可学习提示增强多模态特征对齐，在Charades-STA和ActivityNet-Captions数据集上验证了有效性。
摘要：
时间句子定位涉及使用自然语言查询检索视频时刻。许多现有工作直接结合给定的视频和时间局部化查询进行时间定位，忽略了不同模态之间固有的领域差距。本文利用包含源自相同视频-查询对的广泛时间全局文本知识的伪查询特征，以增强领域差距的弥合，并实现多模态特征之间更高程度的相似性。具体而言，我们提出了伪查询中介网络（PIN），通过对比学习在特征空间中实现视觉与全面伪查询特征的改进对齐。随后，我们利用可学习提示封装伪查询的知识，将其传播至文本编码器和多模态融合模块，进一步增强视觉与语言之间的特征对齐，以实现更好的时间定位。在Charades-STA和ActivityNet-Captions数据集上进行的大量实验证明了我们方法的有效性。
英文摘要：
Temporal sentence grounding involves the retrieval of a video moment with a natural language query. Many existing works directly incorporate the given video and temporally localized query for temporal grounding, overlooking the inherent domain gap between different modalities. In this paper, we utilize pseudo-query features containing extensive temporally global textual knowledge sourced from the same video-query pair, to enhance the bridging of domain gaps and attain a heightened level of similarity between multi-modal features. Specifically, we propose a Pseudo-query Intermediary Network (PIN) to achieve an improved alignment of visual and comprehensive pseudo-query features within the feature space through contrastive learning. Subsequently, we utilize learnable prompts to encapsulate the knowledge of pseudo-queries, propagating them into the textual encoder and multi-modal fusion module, further enhancing the feature alignment between visual and language for better temporal grounding. Extensive experiments conducted on the Charades-STA and ActivityNet-Captions datasets demonstrate the effectiveness of our method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
