DeBiFormer：基于可变形代理双层路由注意力的视觉Transformer
DeBiFormer: Vision Transformer with Deformable Agent Bi-level Routing Attention
AI总结：
提出可变形双层路由注意力模块（DBRA），通过代理查询优化键值对选择并提升可解释性，构建新型通用视觉Transformer DeBiFormer，在图像分类、目标检测和语义分割等任务上验证了其有效性。
摘要：
具有各种注意力模块的视觉Transformer在视觉任务中展现出了卓越的性能。虽然使用如DAT中的稀疏自适应注意力在图像分类中取得了强劲的结果，但在微调语义分割任务时，由可变形点选择的键值对缺乏语义相关性。BiFormer中的查询感知稀疏注意力试图让每个查询聚焦于top-k路由区域。然而，在注意力计算期间，选定的键值对受到过多无关查询的影响，降低了对更重要查询的注意力。为了解决这些问题，我们提出了可变形双层路由注意力（DBRA）模块，该模块使用代理查询优化键值对的选择，并增强了查询在注意力图中的可解释性。基于此，我们引入了DeBiFormer，一种用DBRA模块构建的新型通用视觉Transformer。DeBiFormer已在图像分类、目标检测和语义分割等多种计算机视觉任务上得到验证，为其有效性提供了有力证据。代码可在{https://github.com/maclong01/DeBiFormer}获取。
英文摘要：
Vision Transformers with various attention modules have demonstrated superior performance on vision tasks. While using sparsity-adaptive attention, such as in DAT, has yielded strong results in image classification, the key-value pairs selected by deformable points lack semantic relevance when fine-tuning for semantic segmentation tasks. The query-aware sparsity attention in BiFormer seeks to focus each query on top-k routed regions. However, during attention calculation, the selected key-value pairs are influenced by too many irrelevant queries, reducing attention on the more important ones. To address these issues, we propose the Deformable Bi-level Routing Attention (DBRA) module, which optimizes the selection of key-value pairs using agent queries and enhances the interpretability of queries in attention maps. Based on this, we introduce the Deformable Bi-level Routing Attention Transformer (DeBiFormer), a novel general-purpose vision transformer built with the DBRA module. DeBiFormer has been validated on various computer vision tasks, including image classification, object detection, and semantic segmentation, providing strong evidence of its effectiveness.Code is available at {https://github.com/maclong01/DeBiFormer}
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
