LocalMamba：具有窗口选择性扫描的视觉状态空间模型
LocalMamba: Visual State Space Model with Windowed Selective Scan
AI总结：
本文提出LocalMamba，通过窗口选择性扫描优化视觉Mamba的序列建模，动态搜索每层最优扫描方向，在ImageNet上以相同FLOPs比Vim-Ti提升3.1%。
摘要：
近期在状态空间模型方面的进展，特别是Mamba，在语言理解等任务的长序列建模中展现出了显著的进步。然而，它们在视觉任务中的应用并未显著超越传统卷积神经网络（CNNs）和视觉变换器（ViTs）的性能。本文认为，增强视觉Mamba（ViM）的关键在于优化序列建模的扫描方向。传统的ViM方法将空间令牌展平，忽视了局部二维依赖性的保留，从而拉长了相邻令牌之间的距离。我们引入了一种新颖的局部扫描策略，将图像划分为不同的窗口，有效捕获局部依赖性的同时保持全局视角。此外，考虑到不同网络层对扫描模式的不同偏好，我们提出了一种动态方法，为每一层独立搜索最优扫描选择，显著提升了性能。在普通模型和分层模型上的大量实验均证明了我们的方法在有效捕获图像表示方面的优越性。例如，我们的模型在相同的1.5G FLOPs下，在ImageNet上比Vim-Ti显著高出3.1%。代码可在 https://github.com/hunto/LocalMamba 获取。
英文摘要：
Recent advancements in state space models, notably Mamba, have demonstrated significant progress in modeling long sequences for tasks like language understanding. Yet, their application in vision tasks has not markedly surpassed the performance of traditional Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). This paper posits that the key to enhancing Vision Mamba (ViM) lies in optimizing scan directions for sequence modeling. Traditional ViM approaches, which flatten spatial tokens, overlook the preservation of local 2D dependencies, thereby elongating the distance between adjacent tokens. We introduce a novel local scanning strategy that divides images into distinct windows, effectively capturing local dependencies while maintaining a global perspective. Additionally, acknowledging the varying preferences for scan patterns across different network layers, we propose a dynamic method to independently search for the optimal scan choices for each layer, substantially improving performance. Extensive experiments across both plain and hierarchical models underscore our approach's superiority in effectively capturing image representations. For example, our model significantly outperforms Vim-Ti by 3.1% on ImageNet with the same 1.5G FLOPs. Code is available at: https://github.com/hunto/LocalMamba.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
