arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.06597cs.AI
收藏

TVR-Ranking:一个面向不精确查询的排序视频片段检索数据集

TVR-Ranking: A Dataset for Ranked Video Moment Retrieval with Imprecise Queries

Nanyang Technological University(南洋理工大学);SenseTime(商汤科技)

Renjie Liang, Li Li, Chongzhi Zhang, Jing Wang, Xizhou Zhu, Aixin Sun

AI总结:

本文提出排序视频片段检索(RVMR)任务并构建了TVR-Ranking数据集,通过人工标注94,442个查询-片段对的相关性级别,设计了$NDCG@K, IoU\geq μ$评估指标,实验表明该任务为现有模型带来新挑战,推动了多模态搜索研究。

摘要:

在本文中,我们提出了“排序视频片段检索”(RVMR)任务,旨在通过自然语言查询,从视频集合中定位一个匹配片段的排序列表。尽管CV、NLP和IR社区已提出并研究了一些相关任务,但RVMR最能反映片段搜索的实际场景。为推动RVMR研究,我们基于TVR数据集提供的原始视频和现有片段标注,开发了TVR-Ranking数据集。我们的核心贡献是对94,442个查询-片段对的相关性级别进行了人工标注。随后,我们为这一新任务开发了$NDCG@K, IoU\geq μ$评估指标,并进行了实验以评估三个基线模型。实验表明,新的RVMR任务给现有模型带来了新挑战,我们相信这一新数据集将有助于多模态搜索研究。数据集可在\url{https://github.com/Ranking-VMR/TVR-Ranking}获取。

英文摘要:

In this paper, we propose the task of \textit{Ranked Video Moment Retrieval} (RVMR) to locate a ranked list of matching moments from a collection of videos, through queries in natural language. Although a few related tasks have been proposed and studied by CV, NLP, and IR communities, RVMR is the task that best reflects the practical setting of moment search. To facilitate research in RVMR, we develop the TVR-Ranking dataset, based on the raw videos and existing moment annotations provided in the TVR dataset. Our key contribution is the manual annotation of relevance levels for 94,442 query-moment pairs. We then develop the $NDCG@K, IoU\geq μ$ evaluation metric for this new task and conduct experiments to evaluate three baseline models. Our experiments show that the new RVMR task brings new challenges to existing models and we believe this new dataset contributes to the research on multi-modality search. The dataset is available at \url{https://github.com/Ranking-VMR/TVR-Ranking}

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑