BOAT：双侧局部注意力视觉Transformer
BOAT: Bilateral Local Attention Vision Transformer
摘要：
视觉Transformer在许多计算机视觉任务中取得了出色的性能。早期的视觉Transformer（如ViT和DeiT）采用全局自注意，当补丁数量很大时，这在计算上是昂贵的。为了提高效率，最近的视觉Transformer采用局部自注意力机制，其中在局部窗口内计算自注意力。尽管基于窗口的局部自我注意显著提高了效率，但它未能捕捉图像平面中遥远但相似的斑块之间的关系。为了克服图像空间的局部注意力的限制，在本文中，我们进一步利用补丁在特征空间中的局部性。我们使用它们的特征将补丁分组为多个簇，并在每个簇内计算自我注意力。这种特征空间局部注意力有效地捕捉了不同局部窗口中补丁之间的联系，但仍然相关。我们提出了一个双边局部注意力转换器视觉Transformer（BOAT），它集成了特征空间的局部注意力和图像空间的局部注意力。我们进一步将BOAT与Swin和CSWin模型集成，在几个基准数据集上进行的大量实验表明，我们的BOAT-CSWin模型明显且一致地优于现有的最先进的CNN模型和视觉Transformer。
英文摘要：
Vision Transformers achieved outstanding performance in many computer vision tasks. Early Vision Transformers such as ViT and DeiT adopt global self-attention, which is computationally expensive when the number of patches is large. To improve efficiency, recent Vision Transformers adopt local self-attention mechanisms, where self-attention is computed within local windows. Despite the fact that window-based local self-attention significantly boosts efficiency, it fails to capture the relationships between distant but similar patches in the image plane. To overcome this limitation of image-space local attention, in this paper, we further exploit the locality of patches in the feature space. We group the patches into multiple clusters using their features, and self-attention is computed within every cluster. Such feature-space local attention effectively captures the connections between patches across different local windows but still relevant. We propose a Bilateral lOcal Attention vision Transformer (BOAT), which integrates feature-space local attention with image-space local attention. We further integrate BOAT with both Swin and CSWin models, and extensive experiments on several benchmark datasets demonstrate that our BOAT-CSWin model clearly and consistently outperforms existing state-of-the-art CNN models and vision Transformers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
