HieraMamba:通过分层锚点-Mamba池化实现视频时间接地
HieraMamba: Video Temporal Grounding via Hierarchical Anchor-Mamba Pooling
AI总结:
HieraMamba通过分层锚点-Mamba池化结构,在长视频中实现精确的时间定位,解决了视频时间接地中的全局上下文与细粒度时间细节捕捉难题。
摘要:
视频时间接地任务需要在未剪裁视频中定位自然语言查询的起止时间,要求同时捕捉全局上下文和细粒度时间细节。现有方法在长视频中常因过度下采样或固定窗口导致时间忠实度下降。本文提出HieraMamba,一种分层架构,通过Anchor-MambaPooling(AMP)块利用Mamba的选择性扫描生成紧凑的锚点令牌,以多粒度总结视频内容。两个互补的目标,即锚点条件和段池化对比损失,促使锚点保留局部细节同时保持全局判别性。HieraMamba在Ego4D-NLQ、MAD和TACoS数据集上达到新的SOTA,展示了在长未剪裁视频中的精确时间定位能力。
英文摘要:
Video temporal grounding, the task of localizing the start and end times of a natural language query in untrimmed video, requires capturing both global context and fine-grained temporal detail. This challenge is particularly pronounced in long videos, where existing methods often compromise temporal fidelity by over-downsampling or relying on fixed windows. We present HieraMamba, a hierarchical architecture that preserves temporal structure and semantic richness across scales. At its core are Anchor-MambaPooling (AMP) blocks, which utilize Mamba's selective scanning to produce compact anchor tokens that summarize video content at multiple granularities. Two complementary objectives, anchor-conditioned and segment-pooled contrastive losses, encourage anchors to retain local detail while remaining globally discriminative. HieraMamba sets a new state-of-the-art on Ego4D-NLQ, MAD, and TACoS, demonstrating precise, temporally faithful localization in long, untrimmed videos.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。