变换等变一致性学习用于时间句子定位
Transform-Equivariant Consistency Learning for Temporal Sentence Grounding
AI总结：
针对时间句子定位中数据依赖和分布偏差问题，提出等变一致性调节学习框架，通过自监督增强和一致性损失学习判别性表示，在三个数据集上验证了有效性。
摘要：
本文针对时间句子定位（TSG）任务。尽管现有方法在该任务上已取得不错成绩，但它们不仅严重依赖丰富的视频-查询配对数据进行训练，而且容易陷入数据集分布偏差。为缓解这些局限，我们提出一种新颖的等变一致性调节学习（ECRL）框架，以自监督方式为每个视频学习更具判别性的、与查询相关的帧级表示。我们的动机源于查询引导活动的时间边界应在各种视频级变换下被一致预测。具体而言，我们首先在视频的前景和背景片段上设计一系列时空增强，以生成一组合成视频样本。特别地，我们设计了一个自精炼模块来增强增强视频的完整性和平滑性。然后，我们提出一种新颖的自监督一致性损失（SSCL），应用于原始和增强视频，通过最小化两个视频的序列相似性与时间戳距离的先验高斯分布之间的KL散度，来捕获它们不变的查询相关语义。最后，引入一个共享的定位头来预测原始和增强视频的变换等变查询引导片段边界。在三个具有挑战性的数据集（ActivityNet、TACoS和Charades-STA）上的大量实验证明了我们提出的ECRL框架的有效性和效率。
英文摘要：
This paper addresses the temporal sentence grounding (TSG). Although existing methods have made decent achievements in this task, they not only severely rely on abundant video-query paired data for training, but also easily fail into the dataset distribution bias. To alleviate these limitations, we introduce a novel Equivariant Consistency Regulation Learning (ECRL) framework to learn more discriminative query-related frame-wise representations for each video, in a self-supervised manner. Our motivation comes from that the temporal boundary of the query-guided activity should be consistently predicted under various video-level transformations. Concretely, we first design a series of spatio-temporal augmentations on both foreground and background video segments to generate a set of synthetic video samples. In particular, we devise a self-refine module to enhance the completeness and smoothness of the augmented video. Then, we present a novel self-supervised consistency loss (SSCL) applied on the original and augmented videos to capture their invariant query-related semantic by minimizing the KL-divergence between the sequence similarity of two videos and a prior Gaussian distribution of timestamp distance. At last, a shared grounding head is introduced to predict the transform-equivariant query-guided segment boundaries for both the original and augmented videos. Extensive experiments on three challenging datasets (ActivityNet, TACoS, and Charades-STA) demonstrate both effectiveness and efficiency of our proposed ECRL framework.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
