arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2209.01620cs.CV
收藏

MAFormer:基于多尺度注意力融合的Transformer网络视觉识别

MAFormer: A Transformer Network with Multi-scale Attention Fusion for Visual Recognition

Baidu Inc(百度公司);Beihang University(北京航空航天大学)

Yunhao Wang, Huixin Sun, Xiaodi Wang, Bin Zhang, Chao Li, Ying Xin, Baochang Zhang, Errui Ding, Shumin Han

摘要:

视觉Transformer及其变体在各种计算机视觉任务中表现出巨大的潜力。但是传统的视觉Transformer通常集中在粗糙级别的全局依赖性上,这在令牌级别上遭受全局关系和细粒度表示的学习挑战。在本文中,我们引入多尺度注意力融合到Transformer(MAFormer),它探讨了本地聚合和全球特征提取在视觉识别的双流框架。我们开发了一个简单但有效的模块,通过在令牌级别学习细粒度和粗粒度特征并动态融合它们,来探索Transformers在视觉表示方面的全部潜力。我们的多尺度注意力融合(MAF)模块包括:i)局部窗口注意力分支,学习窗口内的短程交互,聚合细粒度的局部特征; ii)全局特征提取,通过新颖的全局下采样学习(GLD)操作,有效地捕获整个图像内的长程上下文信息; iii)通过注意力自我探索两个特征的整合的融合模块。我们的MAFormer在常见的视觉任务上实现了最先进的性能。特别是,MAFormer-L在ImageNet上达到了85.9 $\%$ Top-1的准确度,分别超过CSWin-B和LV-ViT-L 1.7 $\%$和0.6 $\%$。在MSCOCO上,MAFormer在目标检测上优于现有技术CSWin 1.7个$\%$ mAP,并且在具有类似大小的参数的实例分割上优于现有技术CSWin 1.4个$\%$,从而证明了成为通用骨干网络的潜力。

英文摘要:

Vision Transformer and its variants have demonstrated great potential in various computer vision tasks. But conventional vision transformers often focus on global dependency at a coarse level, which suffer from a learning challenge on global relationships and fine-grained representation at a token level. In this paper, we introduce Multi-scale Attention Fusion into transformer (MAFormer), which explores local aggregation and global feature extraction in a dual-stream framework for visual recognition. We develop a simple but effective module to explore the full potential of transformers for visual representation by learning fine-grained and coarse-grained features at a token level and dynamically fusing them. Our Multi-scale Attention Fusion (MAF) block consists of: i) a local window attention branch that learns short-range interactions within windows, aggregating fine-grained local features; ii) global feature extraction through a novel Global Learning with Down-sampling (GLD) operation to efficiently capture long-range context information within the whole image; iii) a fusion module that self-explores the integration of both features via attention. Our MAFormer achieves state-of-the-art performance on common vision tasks. In particular, MAFormer-L achieves 85.9$\%$ Top-1 accuracy on ImageNet, surpassing CSWin-B and LV-ViT-L by 1.7$\%$ and 0.6$\%$ respectively. On MSCOCO, MAFormer outperforms the prior art CSWin by 1.7$\%$ mAPs on object detection and 1.4$\%$ on instance segmentation with similar-sized parameters, demonstrating the potential to be a general backbone network.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑