用于部分监督时序句子定位的对比统一框架
Contrast-Unity for Partially-Supervised Temporal Sentence Grounding
AI总结：
针对时序句子定位中全监督标注昂贵、弱监督性能差的问题，提出部分监督设置下的对比统一两阶段框架，通过四元组对比学习生成伪标签并训练全监督模型，在两个基准上取得优越性能。
摘要：
时序句子定位旨在从给定的未裁剪视频中检测自然语言查询所描述的事件时间戳。现有的全监督设置取得了优异结果，但需要昂贵的标注成本；而弱监督设置采用廉价标签，性能却较差。为了以更少的标注成本追求高性能，本文引入一种中间的部分监督设置，即训练期间仅有短片段标注可用。为充分利用部分标签，我们专门设计了一个对比统一（contrast-unity）框架，其两阶段目标是隐式到显式的渐进式定位。在隐式阶段，我们使用全面的四元组对比学习（包括事件-查询聚合、事件-背景分离、簇内紧凑性和簇间可分性）在细粒度上对齐事件-查询表示。随后，高质量表示产生可接受的定位伪标签。在显式阶段，为显式优化定位目标，我们利用所得伪标签训练一个全监督模型，以进行定位精化和去噪。在 Charades-STA 和 ActivityNet Captions 上进行的大量实验和充分消融证明了部分监督的重要意义，以及我们方法的优越性能。
英文摘要：
Temporal sentence grounding aims to detect event timestamps described by the natural language query from given untrimmed videos. The existing fully-supervised setting achieves great results but requires expensive annotation costs; while the weakly-supervised setting adopts cheap labels but performs poorly. To pursue high performance with less annotation costs, this paper introduces an intermediate partially-supervised setting, i.e., only short-clip is available during training. To make full use of partial labels, we specially design one contrast-unity framework, with the two-stage goal of implicit-explicit progressive grounding. In the implicit stage, we align event-query representations at fine granularity using comprehensive quadruple contrastive learning: event-query gather, event-background separation, intra-cluster compactness and inter-cluster separability. Then, high-quality representations bring acceptable grounding pseudo-labels. In the explicit stage, to explicitly optimize grounding objectives, we train one fully-supervised model using obtained pseudo-labels for grounding refinement and denoising. Extensive experiments and thoroughly ablations on Charades-STA and ActivityNet Captions demonstrate the significance of partial supervision, as well as our superior performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
