D3G：利用高斯先验进行基于瞥见标注的时间句子定位
D3G: Exploring Gaussian Prior for Temporal Sentence Grounding with Glance Annotation
AI总结：
本文针对时间句子定位提出D3G框架，利用高斯先验与语义一致性采样可靠时刻，并通过动态高斯先验调整缓解瞥见标注偏差，在三个基准上大幅超越弱监督方法并逼近全监督性能。
摘要：
时间句子定位（TSG）旨在根据给定的自然语言查询，从未修剪的视频中定位特定时刻。近年来，弱监督方法仍与全监督方法存在较大性能差距，而后者需要耗费大量人力的时间戳标注。在本研究中，我们旨在降低标注成本，同时在TSG任务上保持与全监督方法相比具有竞争力的性能。为实现这一目标，我们研究了一种最近提出的瞥见监督时间句子定位任务，该任务仅要求为每个查询提供单帧标注（称为瞥见标注）。在该设定下，我们提出了一种基于动态高斯先验的瞥见标注定位框架（D3G），该框架由语义对齐组对比学习模块（SA-GCL）和动态高斯先验调整模块（DGA）组成。具体而言，SA-GCL通过联合利用高斯先验和语义一致性，从二维时间图中采样可靠的正面时刻，从而有助于在联合嵌入空间中对齐正面句子-时刻对。此外，为缓解瞥见标注带来的标注偏差，并对由多个事件组成的复杂查询进行建模，我们提出了DGA模块，该模块动态调整分布以逼近目标时刻的真实分布。在三个具有挑战性的基准上的大量实验验证了所提出的D3G的有效性。它以较大幅度超越了最先进的弱监督方法，并缩小了与全监督方法相比的性能差距。代码可在 https://github.com/solicucu/D3G 获取。
英文摘要：
Temporal sentence grounding (TSG) aims to locate a specific moment from an untrimmed video with a given natural language query. Recently, weakly supervised methods still have a large performance gap compared to fully supervised ones, while the latter requires laborious timestamp annotations. In this study, we aim to reduce the annotation cost yet keep competitive performance for TSG task compared to fully supervised ones. To achieve this goal, we investigate a recently proposed glance-supervised temporal sentence grounding task, which requires only single frame annotation (referred to as glance annotation) for each query. Under this setup, we propose a Dynamic Gaussian prior based Grounding framework with Glance annotation (D3G), which consists of a Semantic Alignment Group Contrastive Learning module (SA-GCL) and a Dynamic Gaussian prior Adjustment module (DGA). Specifically, SA-GCL samples reliable positive moments from a 2D temporal map via jointly leveraging Gaussian prior and semantic consistency, which contributes to aligning the positive sentence-moment pairs in the joint embedding space. Moreover, to alleviate the annotation bias resulting from glance annotation and model complex queries consisting of multiple events, we propose the DGA module, which adjusts the distribution dynamically to approximate the ground truth of target moments. Extensive experiments on three challenging benchmarks verify the effectiveness of the proposed D3G. It outperforms the state-of-the-art weakly supervised methods by a large margin and narrows the performance gap compared to fully supervised methods. Code is available at https://github.com/solicucu/D3G.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
