见、排、滤：通过场景理解的重要词感知Clip过滤用于片段检索和亮点检测
See, Rank, and Filter: Important Word-Aware Clip Filtering via Scene Understanding for Moment Retrieval and Highlight Detection
AI总结：
本文提出通过识别查询中的重要词，结合多模态大语言模型实现视频片段检索和亮点检测的细粒度过滤方法，提升检索和检测性能。
摘要：
视频片段检索（MR）和亮点检测（HD）旨在通过自然语言查询定位视频片段中的相关时刻和关键亮点。然而，现有方法忽视了单个词的重要性，将整个文本查询和视频片段视为黑箱，阻碍了上下文理解。在本文中，我们提出了一种新的方法，通过识别和优先处理查询中的重要词来实现细粒度的片段过滤。我们的方法通过多模态大语言模型（MLLMs）整合图像-文本场景理解，并增强视频片段的语义理解。我们引入了一个特征增强模块（FEM）来捕捉查询中的重要词，并一个基于排名的过滤模块（RFM）来根据这些重要词与视频片段的相关性迭代细化视频片段。广泛的实验表明，我们的方法显著优于现有最先进的方法，在MR和HD任务中均取得优越性能。我们的代码可在：https://github.com/VisualAIKHU/SRF 上获取。
英文摘要：
Video moment retrieval (MR) and highlight detection (HD) with natural language queries aim to localize relevant moments and key highlights in a video clips. However, existing methods overlook the importance of individual words, treating the entire text query and video clips as a black-box, which hinders contextual understanding. In this paper, we propose a novel approach that enables fine-grained clip filtering by identifying and prioritizing important words in the query. Our method integrates image-text scene understanding through Multimodal Large Language Models (MLLMs) and enhances the semantic understanding of video clips. We introduce a feature enhancement module (FEM) to capture important words from the query and a ranking-based filtering module (RFM) to iteratively refine video clips based on their relevance to these important words. Extensive experiments demonstrate that our approach significantly outperforms existing state-of-the-art methods, achieving superior performance in both MR and HD tasks. Our code is available at: https://github.com/VisualAIKHU/SRF.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
