MamFusion：面向部分相关视频检索的多 Mamba 与时序融合
MamFusion: Multi-Mamba with Temporal Fusion for Partially Relevant Video Retrieval
AI总结：
本文提出 MamFusion 框架，通过多 Mamba 与时序 T-to-V、V-to-T 融合建模长视频状态相关性，提升部分相关视频检索精度，并在大规模数据集上取得最先进性能。
摘要：
部分相关视频检索（PRVR）是多媒体检索领域中一项具有挑战性的任务，旨在识别并检索与给定查询部分相关的未剪辑视频。本文研究长序列视频内容理解，以解决信息冗余问题。利用 Mamba 模块出色的长时状态空间建模能力和线性可扩展性，我们提出了一种面向 PRVR 任务的多 Mamba 时序融合框架（MamFusion）。该框架有效捕捉长时视频内容中的状态相关性，并将其无缝整合到文本-视频相关性理解中，从而提升检索过程。具体而言，我们引入时序 T-to-V 融合和时序 V-to-T 融合，显式建模文本查询与视频片段之间的时序关系，提高上下文感知能力和检索精度。在大规模数据集上进行的大量实验表明，MamFusion 在检索效果上达到了最先进的性能。代码可在以下链接获取：https://github.com/Vision-Multimodal-Lab-HZCU/MamFusion。
英文摘要：
Partially Relevant Video Retrieval (PRVR) is a challenging task in the domain of multimedia retrieval. It is designed to identify and retrieve untrimmed videos that are partially relevant to the provided query. In this work, we investigate long-sequence video content understanding to address information redundancy issues. Leveraging the outstanding long-term state space modeling capability and linear scalability of the Mamba module, we introduce a multi-Mamba module with temporal fusion framework (MamFusion) tailored for PRVR task. This framework effectively captures the state-relatedness in long-term video content and seamlessly integrates it into text-video relevance understanding, thereby enhancing the retrieval process. Specifically, we introduce Temporal T-to-V Fusion and Temporal V-to-T Fusion to explicitly model temporal relationships between text queries and video moments, improving contextual awareness and retrieval accuracy. Extensive experiments conducted on large-scale datasets demonstrate that MamFusion achieves state-of-the-art performance in retrieval effectiveness. Code is available at the link: https://github.com/Vision-Multimodal-Lab-HZCU/MamFusion.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
