arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2204.09303cs.CV
收藏

Attention in Attention:为高效视频分类建模上下文相关性

Attention in Attention: Modeling Context Correlation for Efficient Video Classification

University of Science and Technology of China(中国科学技术大学);Wuhan Research Institute of Posts and Telecommunications(武汉邮电科学研究院);Hefei University of Technology(合肥工业大学);Wuhan Institute of Technology(武汉工程大学)

Yanbin Hao, Shuo Wang, Pei Cao, Xinjian Gao, Tong Xu, Jinmeng Wu, Xiangnan He

摘要:

注意力机制利用视角上下文显著提高了视频分类神经网络的性能。然而,目前对视频注意力的研究通常集中在采用上下文的特定方面(例如,信道、空间/时间或全局上下文)来细化特征,并在计算注意力时忽略它们的潜在相关性。这导致不完全的上下文利用,从而承担有限的性能提高的弱点。为了解决这个问题,本文提出了一种有效的注意力(AIA)方法,用于元素特征细化,研究了将通道上下文插入时空注意力学习模块(称为CinST)及其反向变体(称为STinC)的可行性。具体来说,我们将视频特征上下文实例化为沿特定轴聚合的动态,具有全局平均和最大池化操作。AIA模块的工作流程是,第一注意力块使用一种上下文信息来指导针对另一上下文的第二注意力的选通权重计算。此外,注意单元中的所有计算操作都作用于池维,这导致相当少的计算成本增加($<$ 0.02\%)。为了验证我们的方法,我们密集地将其集成到两个经典的视频网络骨干,并在几个标准的视频分类基准上进行了广泛的实验。我们的AIA源代码可在\url{https://github.com/haoyanbin918/Attention-in-Attention}获得。

英文摘要:

Attention mechanisms have significantly boosted the performance of video classification neural networks thanks to the utilization of perspective contexts. However, the current research on video attention generally focuses on adopting a specific aspect of contexts (e.g., channel, spatial/temporal, or global context) to refine the features and neglects their underlying correlation when computing attentions. This leads to incomplete context utilization and hence bears the weakness of limited performance improvement. To tackle the problem, this paper proposes an efficient attention-in-attention (AIA) method for element-wise feature refinement, which investigates the feasibility of inserting the channel context into the spatio-temporal attention learning module, referred to as CinST, and also its reverse variant, referred to as STinC. Specifically, we instantiate the video feature contexts as dynamics aggregated along a specific axis with global average and max pooling operations. The workflow of an AIA module is that the first attention block uses one kind of context information to guide the gating weights calculation of the second attention that targets at the other context. Moreover, all the computational operations in attention units act on the pooled dimension, which results in quite few computational cost increase ($<$0.02\%). To verify our method, we densely integrate it into two classical video network backbones and conduct extensive experiments on several standard video classification benchmarks. The source code of our AIA is available at \url{https://github.com/haoyanbin918/Attention-in-Attention}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑