arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2305.14768cs.CV
收藏

具有分区注意力的双路径Transformer

Dual Path Transformer with Partition Attention

Tencent Youtu Lab(腾讯优图实验室)

Zhengkai Jiang, Liang Liu, Jiangning Zhang, Yabiao Wang, Mingang Chen, Chengjie Wang

AI总结:

本文提出DualFormer,通过结合CNN局部注意力与ViT远程注意力的双注意力机制,并引入多头分区注意力MHPA以降低计算复杂度,在ImageNet、COCO和Cityscapes上取得领先性能。

摘要:

本文介绍了一种新颖的注意力机制,称为双注意力(dual attention),它既高效又有效。双注意力机制由两个并行组件组成:由卷积神经网络(CNNs)生成的局部注意力和由视觉Transformer(ViTs)生成的远程注意力。为了解决原始多头自注意力(MHSA)的高计算复杂度和内存占用问题,我们引入了一种新颖的多头分区注意力(MHPA)机制。分区注意力方法同时建模分区内注意力和分区间注意力。基于双注意力块和分区注意力机制,我们提出了一种名为DualFormer的分层视觉骨干网络。我们在多个计算机视觉任务上评估了模型的有效性,包括ImageNet上的图像分类、COCO上的目标检测以及Cityscapes上的语义分割。具体而言,所提出的DualFormer-XS在ImageNet上实现了81.5%的top-1准确率,比最近的最先进模型MPViT-XS高出0.6%的top-1准确率,同时具有更高的吞吐量。

英文摘要:

This paper introduces a novel attention mechanism, called dual attention, which is both efficient and effective. The dual attention mechanism consists of two parallel components: local attention generated by Convolutional Neural Networks (CNNs) and long-range attention generated by Vision Transformers (ViTs). To address the high computational complexity and memory footprint of vanilla Multi-Head Self-Attention (MHSA), we introduce a novel Multi-Head Partition-wise Attention (MHPA) mechanism. The partition-wise attention approach models both intra-partition and inter-partition attention simultaneously. Building on the dual attention block and partition-wise attention mechanism, we present a hierarchical vision backbone called DualFormer. We evaluate the effectiveness of our model on several computer vision tasks, including image classification on ImageNet, object detection on COCO, and semantic segmentation on Cityscapes. Specifically, the proposed DualFormer-XS achieves 81.5\% top-1 accuracy on ImageNet, outperforming the recent state-of-the-art MPViT-XS by 0.6\% top-1 accuracy with much higher throughput.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑