MambaVision：一种混合Mamba-Transformer视觉骨干网络
MambaVision: A Hybrid Mamba-Transformer Vision Backbone
AI总结：
提出混合Mamba-Transformer视觉骨干MambaVision，通过重新设计Mamba并集成自注意力，在ImageNet-1K分类及COCO、ADE20K下游任务中达到SOTA性能。
摘要：
我们提出了一种新颖的混合Mamba-Transformer骨干网络MambaVision，专门针对视觉应用进行定制。我们的核心贡献包括重新设计Mamba公式，以增强其高效建模视觉特征的能力。通过全面的消融研究，我们证明了将视觉Transformer（ViT）与Mamba集成的可行性。我们的结果表明，在最后几层为Mamba架构配备自注意力块，能显著提高其捕获长距离空间依赖的能力。基于这些发现，我们引入了一系列具有层次结构的MambaVision模型，以满足各种设计标准。在ImageNet-1K数据集上的分类任务中，MambaVision变体在Top-1准确率和吞吐量方面均达到了最先进的（SOTA）性能。在下游任务如MS COCO和ADE20K数据集上的目标检测、实例分割和语义分割中，MambaVision在性能上优于同等规模的骨干网络，同时展现出良好的表现。代码：https://github.com/NVlabs/MambaVision
英文摘要：
We propose a novel hybrid Mamba-Transformer backbone, MambaVision, specifically tailored for vision applications. Our core contribution includes redesigning the Mamba formulation to enhance its capability for efficient modeling of visual features. Through a comprehensive ablation study, we demonstrate the feasibility of integrating Vision Transformers (ViT) with Mamba. Our results show that equipping the Mamba architecture with self-attention blocks in the final layers greatly improves its capacity to capture long-range spatial dependencies. Based on these findings, we introduce a family of MambaVision models with a hierarchical architecture to meet various design criteria. For classification on the ImageNet-1K dataset, MambaVision variants achieve state-of-the-art (SOTA) performance in terms of both Top-1 accuracy and throughput. In downstream tasks such as object detection, instance segmentation, and semantic segmentation on MS COCO and ADE20K datasets, MambaVision outperforms comparably sized backbones while demonstrating favorable performance. Code: https://github.com/NVlabs/MambaVision
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
