VCMamba：融合卷积与多向Mamba的高效视觉表征方法
VCMamba: Bridging Convolutions with Multi-Directional Mamba for Efficient Visual Representation
AI总结：
针对现有视觉模型难以兼顾局部细粒度特征与全局建模能力的问题，提出融合卷积与多向Mamba的分层视觉骨干VCMamba，在图像分类和语义分割任务上以更少参数实现了更优性能。
摘要：
视觉Transformer（ViTs）和状态空间模型（SSMs）的最新进展对卷积神经网络（CNNs）在计算机视觉领域的主导地位发起了挑战。ViTs擅长捕捉全局上下文，Mamba这类SSM对长序列具备线性复杂度，但它们在细粒度局部特征捕捉上不如CNNs有效。反之，CNNs对局部特征有强归纳偏置，却缺乏Transformer和Mamba的全局推理能力。为弥补这一差距，我们提出VCMamba，这是一种融合CNNs与多向Mamba SSM优势的新型视觉骨干网络。VCMamba采用卷积干（convolutional stem）和分层结构，早期阶段通过卷积块提取丰富的局部特征；这些卷积块的输出随后进入后期阶段，由多向Mamba块处理，以高效建模长距离依赖和全局上下文。这种混合设计实现了更优的特征表征，同时保持与图像分辨率相关的线性复杂度。我们通过在ImageNet-1K图像分类和ADE20K语义分割任务上的大量实验验证了VCMamba的有效性。VCMamba-B在ImageNet-1K上取得82.6%的top-1准确率，以少37%的参数超越PlainMamba-L3 0.3个百分点，以少64%的参数超越Vision GNN-B 0.3个百分点；此外，VCMamba-B在ADE20K上取得47.1 mIoU，以少62%的参数比EfficientFormer-L7高出2.0 mIoU。代码可在https://github.com/Wertyuui345/VCMamba获取。
英文摘要：
Recent advances in Vision Transformers (ViTs) and State Space Models (SSMs) have challenged the dominance of Convolutional Neural Networks (CNNs) in computer vision. ViTs excel at capturing global context, and SSMs like Mamba offer linear complexity for long sequences, yet they do not capture fine-grained local features as effectively as CNNs. Conversely, CNNs possess strong inductive biases for local features but lack the global reasoning capabilities of transformers and Mamba. To bridge this gap, we introduce \textit{VCMamba}, a novel vision backbone that integrates the strengths of CNNs and multi-directional Mamba SSMs. VCMamba employs a convolutional stem and a hierarchical structure with convolutional blocks in its early stages to extract rich local features. These convolutional blocks are then processed by later stages incorporating multi-directional Mamba blocks designed to efficiently model long-range dependencies and global context. This hybrid design allows for superior feature representation while maintaining linear complexity with respect to image resolution. We demonstrate VCMamba's effectiveness through extensive experiments on ImageNet-1K classification and ADE20K semantic segmentation. Our VCMamba-B achieves 82.6% top-1 accuracy on ImageNet-1K, surpassing PlainMamba-L3 by 0.3% with 37% fewer parameters, and outperforming Vision GNN-B by 0.3% with 64% fewer parameters. Furthermore, VCMamba-B obtains 47.1 mIoU on ADE20K, exceeding EfficientFormer-L7 by 2.0 mIoU while utilizing 62% fewer parameters. Code is available at https://github.com/Wertyuui345/VCMamba.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
