StableMamba：面向图像与视频的大型状态空间模型的无蒸馏扩展
StableMamba: Distillation-free Scaling of Large SSMs for Images and Videos
AI总结：
本文提出Mamba-Attention交错架构，解决Mamba模型在图像与视频任务中的参数扩展难题，无需知识蒸馏，在多个基准上提升准确率最高达+1.7。
摘要：
状态空间模型（SSMs），以S4为代表，通过将状态空间技术集成到深度学习中，引入了一种新颖的上下文建模方法。然而，由于其数据无关矩阵，它们在全局上下文建模方面存在困难。Mamba模型通过S6选择性扫描算法采用数据相关变体解决了这一问题，增强了上下文建模能力，尤其适用于长序列。然而，基于Mamba的架构在参数数量方面难以扩展，这是视觉应用的一个主要限制。本文解决了大型SSMs在图像分类和动作识别中的可扩展性问题，无需知识蒸馏等额外技术。我们分析了基于Mamba和基于注意力模型的独特特征，提出了一种Mamba-Attention交错架构，以增强可扩展性、鲁棒性和性能。我们证明了这种稳定且高效的交错架构解决了基于Mamba架构在图像和视频中的可扩展性问题，并增强了对JPEG压缩等常见伪影的鲁棒性。我们在ImageNet-1K、Kinetics-400和Something-Something-v2基准上的全面评估表明，我们的方法将最先进的基于Mamba架构的准确率提升了高达+1.7。
英文摘要：
State-space models (SSMs), exemplified by S4, have introduced a novel context modeling method by integrating state-space techniques into deep learning. However, they struggle with global context modeling due to their data-independent matrices. The Mamba model addressed this with data-dependent variants via the S6 selective-scan algorithm, enhancing context modeling, especially for long sequences. However, Mamba-based architectures are difficult to scale with respect to the number of parameters, which is a major limitation for vision applications. This paper addresses the scalability issue of large SSMs for image classification and action recognition without requiring additional techniques like knowledge distillation. We analyze the distinct characteristics of Mamba-based and Attention-based models, proposing a Mamba-Attention interleaved architecture that enhances scalability, robustness, and performance. We demonstrate that the stable and efficient interleaved architecture resolves the scalability issue of Mamba-based architectures for images and videos and increases robustness to common artifacts like JPEG compression. Our thorough evaluation on the ImageNet-1K, Kinetics-400 and Something-Something-v2 benchmarks demonstrates that our approach improves the accuracy of state-of-the-art Mamba-based architectures by up to $+1.7$.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
