PSLT：一种具有阶梯自注意力和渐进式偏移的轻量级视觉Transformer
PSLT: A Light-weight Vision Transformer with Ladder Self-Attention and Progressive Shift
AI总结：
提出轻量级视觉Transformer主干网络PSLT，通过多分支阶梯自注意力块与渐进式偏移机制降低计算成本并扩大感受野，在多项视觉任务上以极低参数量达到与大型模型相当的性能。
摘要：
Vision Transformer (ViT) 因其建模长距离依赖的能力而在各种视觉任务中展现出巨大潜力。然而，ViT 需要大量的计算资源来计算全局自注意力。在本研究中，我们提出了一种具有多分支的阶梯自注意力块和渐进式偏移机制，以开发一种需要较少计算资源（例如相对较少的参数量和 FLOPs）的轻量级 Transformer 主干网络，称为 Progressive Shift Ladder Transformer (PSLT)。首先，阶梯自注意力块通过在每个分支中建模局部自注意力来降低计算成本。同时，提出渐进式偏移机制，通过为每个分支建模多样化的局部自注意力并在这些分支之间进行交互，来扩大阶梯自注意力块中的感受野。其次，阶梯自注意力块的输入特征沿通道维度为每个分支进行均等拆分，这大大降低了阶梯自注意力块的计算成本（参数量和 FLOPs 近似为原来的 1/3），然后通过像素自适应融合协作这些分支的输出。因此，具有相对较少参数量和 FLOPs 的阶梯自注意力块能够建模长距离交互。基于阶梯自注意力块，PSLT 在多项视觉任务上表现良好，包括图像分类、目标检测和行人重识别。在 ImageNet-1k 数据集上，PSLT 取得了 79.9% 的 top-1 准确率，参数量为 9.2M，FLOPs 为 1.9G，这与几个参数量超过 20M 且 FLOPs 超过 4G 的现有模型相当。代码可在 https://isee-ai.cn/wugaojie/PSLT.html 获取。
英文摘要：
Vision Transformer (ViT) has shown great potential for various visual tasks due to its ability to model long-range dependency. However, ViT requires a large amount of computing resource to compute the global self-attention. In this work, we propose a ladder self-attention block with multiple branches and a progressive shift mechanism to develop a light-weight transformer backbone that requires less computing resources (e.g. a relatively small number of parameters and FLOPs), termed Progressive Shift Ladder Transformer (PSLT). First, the ladder self-attention block reduces the computational cost by modelling local self-attention in each branch. In the meanwhile, the progressive shift mechanism is proposed to enlarge the receptive field in the ladder self-attention block by modelling diverse local self-attention for each branch and interacting among these branches. Second, the input feature of the ladder self-attention block is split equally along the channel dimension for each branch, which considerably reduces the computational cost in the ladder self-attention block (with nearly 1/3 the amount of parameters and FLOPs), and the outputs of these branches are then collaborated by a pixel-adaptive fusion. Therefore, the ladder self-attention block with a relatively small number of parameters and FLOPs is capable of modelling long-range interactions. Based on the ladder self-attention block, PSLT performs well on several vision tasks, including image classification, objection detection and person re-identification. On the ImageNet-1k dataset, PSLT achieves a top-1 accuracy of 79.9% with 9.2M parameters and 1.9G FLOPs, which is comparable to several existing models with more than 20M parameters and 4G FLOPs. Code is available at https://isee-ai.cn/wugaojie/PSLT.html.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
