arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.10935cs.CV
收藏

视觉状态空间模型在图像分类中的鲁棒性理解

Understanding Robustness of Visual State Space Models for Image Classification

School of Computer Science, University of Sydney(悉尼大学计算机科学学院)

Chengbin Du, Yanxi Li, Chang Xu

AI总结:

本文系统研究视觉状态空间模型VMamba的鲁棒性,发现其对抗鲁棒性优于Transformer但存在可扩展性弱点,并揭示其新颖组件的脆弱性与防御能力。

摘要:

视觉状态空间模型(VMamba)近期作为一种有前景的架构出现,在各种计算机视觉任务中展现出卓越的性能。然而,其鲁棒性尚未得到深入研究。本文通过多视角的综合调查,深入探讨了该架构的鲁棒性。首先,我们研究了其对对抗攻击的鲁棒性,采用了整图像和补丁特定的对抗攻击。结果表明,与Transformer架构相比,VMamba具有优越的对抗鲁棒性,同时揭示了可扩展性弱点。其次,评估了VMamba在多种场景下的通用鲁棒性,包括自然对抗样本、分布外数据和常见损坏。VMamba在分布外数据上表现出卓越的泛化能力,但在自然对抗样本和常见损坏方面显示出可扩展性弱点。此外,我们探讨了白盒攻击期间VMamba的梯度和反向传播,揭示了其新颖组件的独特脆弱性和防御能力。最后,检查了VMamba对图像结构变化的敏感性,强调了与扰动区域分布和空间信息相关的脆弱性,且越靠近图像中心越易受影响。通过这些综合研究,我们有助于更深入地理解VMamba的鲁棒性,为在计算机视觉应用中改进和推进深度神经网络的能力提供宝贵见解。

英文摘要:

Visual State Space Model (VMamba) has recently emerged as a promising architecture, exhibiting remarkable performance in various computer vision tasks. However, its robustness has not yet been thoroughly studied. In this paper, we delve into the robustness of this architecture through comprehensive investigations from multiple perspectives. Firstly, we investigate its robustness to adversarial attacks, employing both whole-image and patch-specific adversarial attacks. Results demonstrate superior adversarial robustness compared to Transformer architectures while revealing scalability weaknesses. Secondly, the general robustness of VMamba is assessed against diverse scenarios, including natural adversarial examples, out-of-distribution data, and common corruptions. VMamba exhibits exceptional generalizability with out-of-distribution data but shows scalability weaknesses against natural adversarial examples and common corruptions. Additionally, we explore VMamba's gradients and back-propagation during white-box attacks, uncovering unique vulnerabilities and defensive capabilities of its novel components. Lastly, the sensitivity of VMamba to image structure variations is examined, highlighting vulnerabilities associated with the distribution of disturbance areas and spatial information, with increased susceptibility closer to the image center. Through these comprehensive studies, we contribute to a deeper understanding of VMamba's robustness, providing valuable insights for refining and advancing the capabilities of deep neural networks in computer vision applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑