SCANet:用于弱监督视频时刻检索的场景复杂度感知网络
SCANet: Scene Complexity Aware Network for Weakly-Supervised Video Moment Retrieval
AI总结:
针对弱监督视频时刻检索中现有提案未考虑视频场景数量变化的问题,提出SCANet,通过测量视频场景复杂度生成自适应提案,在三个基准上取得最先进性能。
摘要:
视频时刻检索旨在定位视频中与给定语言查询对应的时刻。为避免标注时间时刻的高昂成本,弱监督视频时刻检索(wsVMR)系统已被研究。对于这类系统,生成大量候选时刻提案并选择最合适的提案是一种流行方法,这些提案被假设包含视频中许多可区分的场景作为候选。然而,现有wsVMR系统的提案未考虑每个视频中场景数量的变化,其提案是启发式确定的,与视频无关。我们认为检索系统应能应对每个视频中场景数量变化带来的复杂度。为此,我们提出场景复杂度感知网络(SCANet),它测量每个视频中多个场景的“场景复杂度”,并生成适应每个视频场景复杂度变化的自适应提案。在Charades-STA、ActivityNet和TVR三个检索基准上的实验结果达到了最先进的性能,证明了融入场景复杂度的有效性。
英文摘要:
Video moment retrieval aims to localize moments in video corresponding to a given language query. To avoid the expensive cost of annotating the temporal moments, weakly-supervised VMR (wsVMR) systems have been studied. For such systems, generating a number of proposals as moment candidates and then selecting the most appropriate proposal has been a popular approach. These proposals are assumed to contain many distinguishable scenes in a video as candidates. However, existing proposals of wsVMR systems do not respect the varying numbers of scenes in each video, where the proposals are heuristically determined irrespective of the video. We argue that the retrieval system should be able to counter the complexities caused by varying numbers of scenes in each video. To this end, we present a novel concept of a retrieval system referred to as Scene Complexity Aware Network (SCANet), which measures the `scene complexity' of multiple scenes in each video and generates adaptive proposals responding to variable complexities of scenes in each video. Experimental results on three retrieval benchmarks (i.e., Charades-STA, ActivityNet, TVR) achieve state-of-the-art performances and demonstrate the effectiveness of incorporating the scene complexity.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。