基于正样本挖掘的弱监督时序句子定位
Weakly Supervised Temporal Sentence Grounding via Positive Sample Mining
AI总结：
针对弱监督时序句子定位中相似样本被误作负样本的问题，提出 PSM 框架挖掘语义相似正样本，并结合对比损失与排序损失增强判别监督。
摘要：
弱监督时序句子定位（WSTSG）任务旨在仅利用视频级别的视频-语言对应关系，从未经裁剪的视频中检测与语言描述相对应的时间区间。对于一个锚点样本，大多数现有方法要么从其他视频中、要么在同一视频内生成负样本，以进行对比学习。然而，一些训练样本与锚点样本高度相似，直接将它们视为负样本会给优化带来困难，并忽视这些相似样本与锚点样本之间的相关性。为解决这一问题，我们提出正样本挖掘（PSM），这是一个新颖框架，可从训练集中挖掘正样本，以提供更具判别力的监督。具体而言，对于给定锚点样本，我们根据其余训练样本文本查询的相似性，将其划分为语义相似和语义不相似的子集。为有效利用这些相关性，我们引入 PSM 引导的对比损失，以确保锚点提议更接近相似样本，并远离不相似样本。此外，我们设计了 PSM 引导的排序损失，以确保相似样本更接近锚点提议，而不是视频内负提议，从而区分锚点提议与视频内负提议。在 WSTSG 和基于定位的 VideoQA 任务上的实验证明了本方法的有效性和优越性。
英文摘要：
The task of weakly supervised temporal sentence grounding (WSTSG) aims to detect temporal intervals corresponding to a language description from untrimmed videos with only video-level video-language correspondence. For an anchor sample, most existing approaches generate negative samples either from other videos or within the same video for contrastive learning. However, some training samples are highly similar to the anchor sample, directly regarding them as negative samples leads to difficulties for optimization and ignores the correlations between these similar samples and the anchor sample. To address this, we propose Positive Sample Mining (PSM), a novel framework that mines positive samples from the training set to provide more discriminative supervision. Specifically, for a given anchor sample, we partition the remaining training set into semantically similar and dissimilar subsets based on the similarity of their text queries. To effectively leverage these correlations, we introduce a PSM-guided contrastive loss to ensure that the anchor proposal is closer to similar samples and further from dissimilar ones. Additionally, we design a PSM-guided rank loss to ensure that similar samples are closer to the anchor proposal than to the negative intra-video proposal, aiming to distinguish the anchor proposal and the negative intra-video proposal. Experiments on the WSTSG and grounded VideoQA tasks demonstrate the effectiveness and superiority of our method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
