无冗余时序视频定位:多尺度邻域注意力与放大边界检测
No-frills Temporal Video Grounding: Multi-Scale Neighboring Attention and Zoom-in Boundary Detection
AI总结:
针对时序视频定位中语义噪声比低导致性能下降的问题,提出含多尺度邻域注意力与放大边界检测的轻量化无冗余模型,在基准上取得有竞争力的性能,且推理更快、参数量更少。
摘要:
时序视频定位(Temporal Video Grounding, TVG)旨在从未修剪视频中检索出与语言查询对应的时间区间。TVG的一大核心挑战是“语义噪声比(Semantic Noise Ratio, SNR)”较低,且SNR越低模型性能越差。现有研究多采用复杂技术应对这一难题。本文提出一种无冗余的TVG模型,包含两个核心模块:多尺度邻域注意力与放大边界检测。多尺度邻域注意力限制每个视频token仅聚合其邻域的视觉上下文,能够在高比例噪声中通过多尺度特征层级提取最具区分度的信息。放大边界检测则聚焦于对筛选出的 top 候选区间进行局部判别,实现细粒度的定位调整。采用端到端训练策略,本文模型在多个TVG基准数据集上取得了具有竞争力的性能,同时凭借轻量化架构,具备推理速度更快、模型参数量更少的优势。
英文摘要:
Temporal video grounding (TVG) aims to retrieve the time interval of a language query from an untrimmed video. A significant challenge in TVG is the low "Semantic Noise Ratio (SNR)", which results in worse performance with lower SNR. Prior works have addressed this challenge using sophisticated techniques. In this paper, we propose a no-frills TVG model that consists of two core modules, namely multi-scale neighboring attention and zoom-in boundary detection. The multi-scale neighboring attention restricts each video token to only aggregate visual contexts from its neighbor, enabling the extraction of the most distinguishing information with multi-scale feature hierarchies from high-ratio noises. The zoom-in boundary detection then focuses on local-wise discrimination of the selected top candidates for fine-grained grounding adjustment. With an end-to-end training strategy, our model achieves competitive performance on different TVG benchmarks, while also having the advantage of faster inference speed and lighter model parameters, thanks to its lightweight architecture.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。