一种灵活且可扩展的视频时刻搜索框架
A Flexible and Scalable Framework for Video Moment Search
AI总结：
针对视频时刻搜索中长视频处理低效和单时刻假设的问题，提出 SPR 三阶段框架，通过片段检索、候选生成与重排序实现任意长度视频的可扩展 RVMR，并在 TVR-Ranking 上取得最优性能。
摘要：
视频时刻搜索是在视频语料库中查找与用户查询相匹配的相关时刻的过程，对多种应用都至关重要。然而，现有解决方案通常假设只存在单一的完美匹配时刻，面临推理效率低下的问题，并且在处理长达数小时的视频时存在局限。本文提出一种灵活且可扩展的框架，用于从任意长度的视频集合中检索与文本查询匹配的时刻排序列表，该任务被称为排序视频时刻检索（Ranked Video Moment Retrieval，RVMR）。我们的框架名为 Segment-Proposal-Ranking（SPR），将搜索过程简化为三个独立阶段：片段检索、候选生成以及带重排序的时刻细化。具体而言，视频被划分为等长片段，并对预先计算的嵌入进行离线索引，从而无论视频长度如何都能实现高效检索。为实现可扩展的在线检索，片段和查询都被投影到共享特征空间，以支持近似最近邻（ANN）搜索。随后，检索到的片段被合并为粗粒度的时刻候选。接着设计了一个细化与重排序模块，用于对粗粒度候选进行重新排序并调整其时间戳。在 TVR-Ranking 数据集上的评估表明，本框架取得了最先进的性能，并显著降低了计算成本和处理时间。这种灵活设计还允许对每个阶段进行独立改进，使 SPR 能够高度适应大规模应用。
英文摘要：
Video moment search, the process of finding relevant moments in a video corpus to match a user's query, is crucial for various applications. Existing solutions, however, often assume a single perfect matching moment, struggle with inefficient inference, and have limitations with hour-long videos. This paper introduces a flexible and scalable framework for retrieving a ranked list of moments from collection of videos in any length to match a text query, a task termed Ranked Video Moment Retrieval (RVMR). Our framework, called Segment-Proposal-Ranking (SPR), simplifies the search process into three independent stages: segment retrieval, proposal generation, and moment refinement with re-ranking. Specifically, videos are divided into equal-length segments with precomputed embeddings indexed offline, allowing efficient retrieval regardless of video length. For scalable online retrieval, both segments and queries are projected into a shared feature space to enable approximate nearest neighbor (ANN) search. Retrieved segments are then merged into coarse-grained moment proposals. Then a refinement and re-ranking module is designed to reorder and adjust timestamps of the coarse-grained proposals. Evaluations on the TVR-Ranking dataset demonstrate that our framework achieves state-of-the-art performance with significant reductions in computational cost and processing time. The flexible design also allows for independent improvements to each stage, making SPR highly adaptable for large-scale applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
