视频语料库时刻检索的选择性查询引导去偏
Selective Query-guided Debiasing for Video Corpus Moment Retrieval
AI总结：
针对视频时刻检索中现有系统依赖检索偏差、未充分学习多模态交互的问题，提出含“有偏时刻检索”和“选择性查询引导去偏”特性的SQuiDNet，在TVR等三个基准上验证了有效性并提升可解释性。
摘要：
视频时刻检索（VMR）旨在在未剪辑视频中定位与给定文本查询相关的目标时刻。现有检索系统往往依赖检索偏差作为捷径，因此未能充分学习查询与视频之间的多模态交互。这种检索偏差源于学习查询与时刻之间的频繁共现模式，该模式将查询中提及的对象（如铅笔）与视频中这些对象频繁出现的时刻（如用铅笔书写的场景）虚假关联，导致模型收敛于有偏的时刻预测。尽管近期去偏方法聚焦于消除这种检索偏差，但我们认为这些有偏预测有时应被保留，因为许多查询中，有偏预测反而有帮助。为结合这种检索偏差，我们提出选择性查询引导去偏网络（SQuiDNet），其包含两个主要特性：（1）有偏时刻检索，有意揭示查询对象中固有的有偏时刻；（2）选择性查询引导去偏，根据查询的含义执行选择性去偏。我们在三个时刻检索基准（即TVR、ActivityNet、DiDeMo）上的实验结果证明了SQuiDNet的有效性，定性分析显示其可解释性得到提升。
英文摘要：
Video moment retrieval (VMR) aims to localize target moments in untrimmed videos pertinent to a given textual query. Existing retrieval systems tend to rely on retrieval bias as a shortcut and thus, fail to sufficiently learn multi-modal interactions between query and video. This retrieval bias stems from learning frequent co-occurrence patterns between query and moments, which spuriously correlate objects (e.g., a pencil) referred in the query with moments (e.g., scene of writing with a pencil) where the objects frequently appear in the video, such that they converge into biased moment predictions. Although recent debiasing methods have focused on removing this retrieval bias, we argue that these biased predictions sometimes should be preserved because there are many queries where biased predictions are rather helpful. To conjugate this retrieval bias, we propose a Selective Query-guided Debiasing network (SQuiDNet), which incorporates the following two main properties: (1) Biased Moment Retrieval that intentionally uncovers the biased moments inherent in objects of the query and (2) Selective Query-guided Debiasing that performs selective debiasing guided by the meaning of the query. Our experimental results on three moment retrieval benchmarks (i.e., TVR, ActivityNet, DiDeMo) show the effectiveness of SQuiDNet and qualitative analysis shows improved interpretability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
