用于改进 Vision Mamba 训练的随机逐层打乱
Stochastic Layer-Wise Shuffle for Improving Vision Mamba Training
AI总结：
本文提出了一种即插即用的正则化方法 Stochastic Layer-Wise Shuffle (SLWS)，通过逐层概率分配与伯努利采样对输入 token 进行随机打乱，在不修改架构的情况下显著提升了 Vision Mamba 在 ImageNet-1K 上的性能。
摘要：
近期的 Vision Mamba (Vim) 模型在序列长度上展现出近乎线性的复杂度，使其在处理视觉数据时极具吸引力。然而，其训练方法及潜力仍未被充分探索。本文研究了 Vim 的策略，并提出了 Stochastic Layer-Wise Shuffle (SLWS)，这是一种能有效改进 Vim 训练的新型正则化方法。无需架构修改，该方法使非分层的 Vim 在 ImageNet-1K 上取得领先性能。该方法在每层通过四个简单步骤运作：通过概率分配设定依赖层的打乱率，通过伯努利试验进行操作采样，对输入 token 序列进行打乱，以及恢复输出的顺序。SLWS 通过三个原则脱颖而出：(1) 即插即用：无需架构修改，且在推理期间停用。(2) 简单而有效：四步过程仅引入随机置换和可忽略的开销。(3) 直观设计：打乱概率随层深度线性增长，符合视觉模型中的分层语义抽象。本研究强调了为 Vim 模型定制训练策略的重要性，并为探索其可扩展性提供了有益途径。
英文摘要：
Recent Vision Mamba (Vim) models exhibit nearly linear complexity in sequence length, making them highly attractive for processing visual data. However, the training methodologies and their potential are still not sufficiently explored. In this paper, we investigate strategies for Vim and propose Stochastic Layer-Wise Shuffle (SLWS), a novel regularization method that can effectively improve the Vim training. Without architectural modifications, this approach enables the non-hierarchical Vim to get leading performance on ImageNet-1K compared with the similar type counterparts. Our method operates through four simple steps per layer: probability allocation to assign layer-dependent shuffle rates, operation sampling via Bernoulli trials, sequence shuffling of input tokens, and order restoration of outputs. SLWS distinguishes itself through three principles: \textit{(1) Plug-and-play:} No architectural modifications are needed, and it is deactivated during inference. \textit{(2) Simple but effective:} The four-step process introduces only random permutations and negligible overhead. \textit{(3) Intuitive design:} Shuffling probabilities grow linearly with layer depth, aligning with the hierarchical semantic abstraction in vision models. Our work underscores the importance of tailored training strategies for Vim models and provides a helpful way to explore their scalability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
