EfficientVMamba:用于轻量级视觉Mamba的空洞选择性扫描
EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba
AI总结:
针对轻量级视觉模型的效率与精度权衡问题,提出EfficientVMamba,通过空洞选择性扫描和高效跳跃采样,结合SSM与卷积分支,在降低计算量的同时提升性能,如S模型以1.3G FLOPs在ImageNet上超越Vim-Ti 5.6%。
摘要:
先前在轻量级模型开发中的努力主要集中在基于CNN和Transformer的设计上,但面临持续的挑战。CNN擅长局部特征提取,但会牺牲分辨率;而Transformer提供全局视野,但计算需求呈二次方增长($\mathcal{O}(N^2)$)。这种在精度和效率之间的持续权衡仍然是一个重大障碍。最近,状态空间模型(SSM),如Mamba,在语言建模和计算机视觉等多项任务中展现出卓越的性能和竞争力,同时将全局信息提取的时间复杂度降低到$\mathcal{O}(N)$。受此启发,本工作旨在探索视觉状态空间模型在轻量级模型设计中的潜力,并引入一种新颖的高效模型变体,称为EfficientVMamba。具体而言,我们的EfficientVMamba通过高效跳跃采样整合了一种基于空洞的选择性扫描方法,构建了旨在同时利用全局和局部表示特征的构建块。此外,我们研究了SSM块与卷积之间的集成,并引入了一种结合额外卷积分支的高效视觉状态空间块,进一步提升了模型性能。实验结果表明,EfficientVMamba在降低计算复杂度的同时,在多种视觉任务上取得了有竞争力的结果。例如,我们的EfficientVMamba-S在1.3G FLOPs下,在ImageNet上以5.6%的准确率大幅超越了1.5G FLOPs的Vim-Ti。代码可在https://github.com/TerryPei/EfficientVMamba获取。
英文摘要:
Prior efforts in light-weight model development mainly centered on CNN and Transformer-based designs yet faced persistent challenges. CNNs adept at local feature extraction compromise resolution while Transformers offer global reach but escalate computational demands $\mathcal{O}(N^2)$. This ongoing trade-off between accuracy and efficiency remains a significant hurdle. Recently, state space models (SSMs), such as Mamba, have shown outstanding performance and competitiveness in various tasks such as language modeling and computer vision, while reducing the time complexity of global information extraction to $\mathcal{O}(N)$. Inspired by this, this work proposes to explore the potential of visual state space models in light-weight model design and introduce a novel efficient model variant dubbed EfficientVMamba. Concretely, our EfficientVMamba integrates a atrous-based selective scan approach by efficient skip sampling, constituting building blocks designed to harness both global and local representational features. Additionally, we investigate the integration between SSM blocks and convolutions, and introduce an efficient visual state space block combined with an additional convolution branch, which further elevate the model performance. Experimental results show that, EfficientVMamba scales down the computational complexity while yields competitive results across a variety of vision tasks. For example, our EfficientVMamba-S with $1.3$G FLOPs improves Vim-Ti with $1.5$G FLOPs by a large margin of $5.6\%$ accuracy on ImageNet. Code is available at: \url{https://github.com/TerryPei/EfficientVMamba}.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。