HERO: 分层嵌入-细化用于视频中开放词汇时间句接地
HERO: Hierarchical Embedding-Refinement for Open-Vocabulary Temporal Sentence Grounding in Videos
AI总结：
HERO提出一种分层嵌入-细化框架，用于视频中开放词汇时间句接地，通过多级语义建模和跨模态细化提升视频与语言的对齐能力。
摘要：
视频中的时间句接地（TSGV）旨在时间上定位与给定自然语言查询对应的视频片段。尽管近年来取得了进展，但大多数现有的TSGV方法仍局限于封闭词汇设置，限制了它们在涉及新奇或多样语言表达的现实查询中的泛化能力。为弥合这一关键差距，我们引入了开放词汇TSGV（OV-TSGV）任务，并构建了首个专用基准——Charades-OV和ActivityNet-OV，模拟现实中的词汇变化和同义词变化。这些基准促进了模型在超出训练概念的系统评估。为解决OV-TSGV问题，我们提出了HERO（分层嵌入-细化用于开放词汇接地），一个统一的框架，利用分层语言嵌入并执行并行跨模态细化。HERO联合建模多级语义并通过语义引导的视觉过滤和对比掩码文本细化增强视频-语言对齐。在标准和开放词汇基准上的广泛实验表明，HERO在开放词汇场景中始终优于最先进的方法，验证了其强大的泛化能力，并突显了OV-TSGV作为新研究方向的重要性。
英文摘要：
Temporal Sentence Grounding in Videos (TSGV) aims to temporally localize segments of a video that correspond to a given natural language query. Despite recent progress, most existing TSGV approaches operate under closed-vocabulary settings, limiting their ability to generalize to real-world queries involving novel or diverse linguistic expressions. To bridge this critical gap, we introduce the Open-Vocabulary TSGV (OV-TSGV) task and construct the first dedicated benchmarks--Charades-OV and ActivityNet-OV--that simulate realistic vocabulary shifts and paraphrastic variations. These benchmarks facilitate systematic evaluation of model generalization beyond seen training concepts. To tackle OV-TSGV, we propose HERO(Hierarchical Embedding-Refinement for Open-Vocabulary grounding), a unified framework that leverages hierarchical linguistic embeddings and performs parallel cross-modal refinement. HERO jointly models multi-level semantics and enhances video-language alignment via semantic-guided visual filtering and contrastive masked text refinement. Extensive experiments on both standard and open vocabulary benchmarks demonstrate that HERO consistently surpasses state-of-the-art methods, particularly under open-vocabulary scenarios, validating its strong generalization capability and underscoring the significance of OV-TSGV as a new research direction.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
