超越基于描述的查询的视频时刻检索
Beyond Caption-Based Queries for Video Moment Retrieval
AI总结：
本文提出了一种改进视频时刻检索方法，通过解决语言和多时刻查询的泛化问题，提升了搜索查询的性能。
摘要：
在本文中，我们研究了在基于描述的查询上训练但用搜索查询评估时现有视频时刻检索（VMR）方法的退化现象，特别是DETR架构。为此，我们通过修改三个公共VMR数据集中的文本查询引入了三个基准——即HD-EPIC、YouCook2和ActivityNet-Captions。我们的分析揭示了两个关键的泛化挑战：（i）语言差距，源于搜索查询语言的不充分指定；（ii）多时刻差距，由于从单时刻查询转向多时刻查询。我们还识别出这些架构中的一个关键问题——活跃解码器查询崩溃——是泛化到多时刻实例的较差表现的主要原因。我们通过架构修改来缓解这个问题，这些修改有效地增加了活跃解码器查询的数量。广泛的实验表明，我们的方法在搜索查询上的性能提高了高达14.82%的mAP_m，在多时刻搜索查询上的性能提高了高达21.83%的mAP_m。代码、模型和数据可在项目网页上获得：https://davidpujol.github.io/beyond-vmr/
英文摘要：
In this work, we investigate the degradation of existing VMR methods, particularly of DETR architectures, when trained on caption-based queries but evaluated on search queries. For this, we introduce three benchmarks by modifying the textual queries in three public VMR datasets -- i.e., HD-EPIC, YouCook2 and ActivityNet-Captions. Our analysis reveals two key generalization challenges: (i) A language gap, arising from the linguistic under-specification of search queries, and (ii) a multi-moment gap, caused by the shift from single-moment to multi-moment queries. We also identify a critical issue in these architectures -- an active decoder-query collapse -- as a primary cause of the poor generalization to multi-moment instances. We mitigate this issue with architectural modifications that effectively increase the number of active decoder queries. Extensive experiments demonstrate that our approach improves performance on search queries by up to 14.82% mAP_m, and up to 21.83% mAP_m on multi-moment search queries. The code, models and data are available in the project webpage: https://davidpujol.github.io/beyond-vmr/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
