We present CSWin Transformer, an efficient and effective Transformer-based backbone for general-purpose vision tasks. A challenging issue in Transformer design is that global self-attention is very expensive to compute whereas local self-attention often limits the field of interactions of each token. To address this issue, we develop the Cross-Shaped Window self-attention mechanism for computing self-attention in the horizontal and vertical stripes in parallel that form a cross-shaped window, with each stripe obtained by splitting the input feature into stripes of equal width. We provide a mathematical analysis of the effect of the stripe width and vary the stripe width for different layers of the Transformer network which achieves strong modeling capability while limiting the computation cost. We also introduce Locally-enhanced Positional Encoding (LePE), which handles the local positional information better than existing encoding schemes. LePE naturally supports arbitrary input resolutions, and is thus especially effective and friendly for downstream tasks. Incorporated with these designs and a hierarchical structure, CSWin Transformer demonstrates competitive performance on common vision tasks. Specifically, it achieves 85.4\% Top-1 accuracy on ImageNet-1K without any extra training data or label, 53.9 box AP and 46.4 mask AP on the COCO detection task, and 52.2 mIOU on the ADE20K semantic segmentation task, surpassing previous state-of-the-art Swin Transformer backbone by +1.2, +2.0, +1.4, and +2.0 respectively under the similar FLOPs setting. By further pretraining on the larger dataset ImageNet-21K, we achieve 87.5% Top-1 accuracy on ImageNet-1K and high segmentation performance on ADE20K with 55.7 mIoU. The code and models are available at https://github.com/microsoft/CSWin-Transformer.
Pale Transformer: A General Vision Transformer Backbone with Pale-Shaped Attention
Institute of Deep Learning, Baidu Research（百度研究院深度学习研究所）；National Engineering Laboratory for Deep Learning Technology and Application（深度学习技术及应用国家工程实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Vision Transformer and its variants have demonstrated great potential in various computer vision tasks. But conventional vision transformers often focus on global dependency at a coarse level, which suffer from a learning challenge on global relationships and fine-grained representation at a token level. In this paper, we introduce Multi-scale Attention Fusion into transformer (MAFormer), which explores local aggregation and global feature extraction in a dual-stream framework for visual recognition. We develop a simple but effective module to explore the full potential of transformers for vis…
PKINet-v2: Towards Powerful and Efficient Poly-Kernel Remote Sensing Object Detection
Nanjing University of Science and Technology（南京理工大学）；Zhejiang University（浙江大学）；Department of Electrical and Computer Engineering, Sungkyunkwan University（成均馆大学电子与计算机工程系）；State Key Lab. of Intelligent Manufacturing of Advanced Construction Machinery（先进施工机械智能制造国家重点实验室）
SAM2-UNet: Segment Anything 2 Makes Strong Encoder for Natural and Medical Image Segmentation
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；School of Computer and Information Engineering, Jiangxi Normal University（江西师范大学计算机与信息工程学院）；The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；Tianjin University（天津大学）；Monash University（墨尔本大学）；Pazhou Lab（琶洲实验室）；Smart Hospital Research Institute, Peking University Shenzhen Hospital（北京大学深圳医院智能医院研究所）
Active Spatial Guidance: Eliminating Injected Positional Mechanisms in Vision Transformers
School of Engineering, Yancheng Institute of Technology（盐城工学院工程学院）；School of Computer Information Engineering, Changzhou Institute of Technology（常州工学院计算机信息工程学院）；Advanced Robotics and Intelligent Systems Laboratory, School of Engineering, University of Guelph（圭尔夫大学工程学院先进机器人与智能系统实验室）
HBFormer: A Hybrid-Bridge Transformer for Microtumor and Miniature Organ Segmentation
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究所，中国科学院）；Guangdong University of Technology（广东技术大学）；University of Macau（澳门大学）；Huizhou University（惠州大学）
U-DiTs: Downsample Tokens in U-Shaped Diffusion Transformers
Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；State Key Lab of General AI, School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院通用人工智能国家重点实验室）
MEVL-STP: Multi-Encoder and Vision Language Model for Arbitrarily Shaped Scene Text Spotting
Rajiv Gandhi Institute of Petroleum Technology（拉吉夫·甘地石油技术学院）；Indian Institute of Technology (ISM) Dhanbad（印度理工学院（ISM）丹巴德分校）；University of Salford（索尔福德大学）
ALICE: Learning a General-Purpose Pathology Foundation Model from Vision, Vision-Language, and Slide-Level Experts
Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学生物医药与健康工程研究院，清华大学深圳国际研究生院）；Department of Engineering Science, University of Oxford（牛津大学工程科学系）；Department of Pathology, The First Affiliated Hospital of Sun Yat-sen University（中山大学附属第一医院病理科）；Medical Optical Technology R&D Center, Research Institute of Tsinghua, Pearl River Delta（清华珠三角研究院医学光学技术研发中心）
