arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2207.06580cs.CVcs.AIcs.LGcs.MM
收藏

基于全局分割模板学习的无命题时间动作检测

Proposal-Free Temporal Action Detection via Global Segmentation Mask Learning

University of Surrey(萨里大学);iFlyTek-Surrey Joint Research Centre on Artificial Intelligence(讯飞-萨里人工智能联合研究中心);Surrey Institute for People-Centred Artificial Intelligence(萨里以人为本人工智能研究所)

Sauradip Nag, Xiatian Zhu, Yi-Zhe Song, Tao Xiang

摘要:

现有的时间动作检测(Temporal Action Detection,简称TACT)方法依赖于为每个视频生成大量的建议。由于提案生成和/或按提案操作实例评估以及由此产生的高计算成本,这会导致复杂的模型设计。在这项工作中,第一次,我们提出了一个建议免费的时间动作检测模型与全球分割掩模(TAGS)。我们的核心思想是在整个视频长度上联合学习每个动作实例的全局分割掩码。TAGS模型与传统的基于提议的方法有很大的不同,它专注于全局时间表示学习,直接检测出没有提议的动作实例的局部开始和结束点。此外,通过整体建模,而不是在个人的建议水平,TAGS需要一个更简单的模型架构,具有较低的计算成本。大量的实验表明,尽管其简单的设计,TAGS优于现有的并行计算方法,实现了新的国家的最先进的性能在两个基准。重要的是,它的训练速度快约20倍,推理效率高约1.6倍。我们的PyTorch实现TAGS可以在https://github.com/sauradip/TAGS上找到。

英文摘要:

Existing temporal action detection (TAD) methods rely on generating an overwhelmingly large number of proposals per video. This leads to complex model designs due to proposal generation and/or per-proposal action instance evaluation and the resultant high computational cost. In this work, for the first time, we propose a proposal-free Temporal Action detection model with Global Segmentation mask (TAGS). Our core idea is to learn a global segmentation mask of each action instance jointly at the full video length. The TAGS model differs significantly from the conventional proposal-based methods by focusing on global temporal representation learning to directly detect local start and end points of action instances without proposals. Further, by modeling TAD holistically rather than locally at the individual proposal level, TAGS needs a much simpler model architecture with lower computational cost. Extensive experiments show that despite its simpler design, TAGS outperforms existing TAD methods, achieving new state-of-the-art performance on two benchmarks. Importantly, it is ~ 20x faster to train and ~1.6x more efficient for inference. Our PyTorch implementation of TAGS is available at https://github.com/sauradip/TAGS .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑