arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2005.06803cs.CV
收藏

TAM:用于视频识别的时间自适应模块

TAM: Temporal Adaptive Module for Video Recognition

State Key Lab for Novel Software Technology, Nanjing University, China(南京大学计算机软件新技术国家重点实验室);SenseTime Research(商汤科技研究院)

Zhaoyang Liu, Limin Wang, Wayne Wu, Chen Qian, Tong Lu

摘要:

由于摄像机运动、速度变化和不同的活动等各种因素,视频数据具有复杂的时间动态。为了有效地捕捉这种不同的运动模式,本文提出了一种新的时间自适应模块({ \bf TAM}),以产生视频特定的时间内核的基础上,自己的特征地图。TAM提出了一种独特的两级自适应建模方案,通过解耦的动态内核到一个位置敏感的重要性地图和位置不变的聚合权重。重要性图是在局部时间窗口中学习的,以捕获短期信息,而聚合权重是从全局视图生成的,重点是长期结构。TAM是一个模块化的块,可以集成到2D CNN中,以非常小的额外计算成本产生强大的视频架构(TANet)。在Kinetics-400和Something-Something数据集上的大量实验表明,该方法的性能始终优于其他时态建模方法,并在相似的复杂度下达到了最先进的性能。代码可在\url{ https://github.com/liu-zhy/temporal-adaptive-module}上获得。

英文摘要:

Video data is with complex temporal dynamics due to various factors such as camera motion, speed variation, and different activities. To effectively capture this diverse motion pattern, this paper presents a new temporal adaptive module ({\bf TAM}) to generate video-specific temporal kernels based on its own feature map. TAM proposes a unique two-level adaptive modeling scheme by decoupling the dynamic kernel into a location sensitive importance map and a location invariant aggregation weight. The importance map is learned in a local temporal window to capture short-term information, while the aggregation weight is generated from a global view with a focus on long-term structure. TAM is a modular block and could be integrated into 2D CNNs to yield a powerful video architecture (TANet) with a very small extra computational cost. The extensive experiments on Kinetics-400 and Something-Something datasets demonstrate that our TAM outperforms other temporal modeling methods consistently, and achieves the state-of-the-art performance under the similar complexity. The code is available at \url{ https://github.com/liu-zhy/temporal-adaptive-module}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑