多提案协作与多任务训练用于弱监督视频时刻检索
Multi-proposal Collaboration and Multi-task Training for Weakly-supervised Video Moment Retrieval
AI总结：
本文提出MCMT方法，通过多提案生成和多任务训练提升弱监督视频时刻检索的鲁棒性与稳定性，实验验证其有效性。
摘要：
本文研究了弱监督视频时刻检索（VMR），旨在仅利用视频级对应关系，在未剪辑的视频中识别出与给定查询语义相似的时刻，而无需依赖时间标注进行训练。先前的方法要么对视频中的所有实例进行预测汇总，要么通过间接方式提出查询的重建来解决该任务。然而，这些方法往往会产生低质量的时间提案，难以区分同一视频中的错位时刻，或由于依赖单一辅助任务而缺乏稳定性。为解决这些限制，我们提出了一种新的弱监督方法，称为多提案协作与多任务训练（MCMT）。首先，我们生成多个提案，并从中推导出相应的可学习高斯掩码。这些掩码随后被结合，以创建高质量的正样本掩码，突出与查询最相关的视频片段。同时，我们将同一视频中的其他片段分类为容易的负样本，将整个视频分类为困难的负样本。在训练过程中，我们引入了正向和反向掩码查询重建任务，以对网络施加更严格的约束，促进更稳健和稳定的检索性能。在两个标准基准上的广泛实验验证了所提出方法在VMR中的有效性。
英文摘要：
This study focuses on weakly-supervised Video Moment Retrieval (VMR), aiming to identify a moment semantically similar to the given query within an untrimmed video using only video-level correspondences, without relying on temporal annotations during training. Previous methods either aggregate predictions for all instances in the video, or indirectly address the task by proposing reconstructions for the query. However, these methods often produce low-quality temporal proposals, struggle with distinguishing misaligned moments in the same video, or lack stability due to a reliance on a single auxiliary task. To address these limitations, we present a novel weakly-supervised method called Multi-proposal Collaboration and Multi-task Training (MCMT). Initially, we generate multiple proposals and derive corresponding learnable Gaussian masks from them. These masks are then combined to create a high-quality positive sample mask, highlighting video clips most relevant to the query. Concurrently, we classify other clips in the same video as the easy negative sample and the entire video as the hard negative sample. During training, we introduce forward and inverse masked query reconstruction tasks to impose more substantial constraints on the network, promoting more robust and stable retrieval performance. Extensive experiments on two standard benchmarks affirm the effectiveness of the proposed method in VMR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
