arXiv 2406.094072026-06-03cs.CV

评估视觉状态空间模型的鲁棒性

Towards Evaluating the Robustness of Visual State Space Models

Mohamed Bin Zayed University of AI（Mohamed Bin Zayed人工智能大学）；Center of Secure Cyber-Physical Security Systems（安全的网络物理安全系统中心）；Linköping University（林波伊大学）；Australian National University（澳大利亚国立大学）

Hashmat Shadab Malik, Fahad Shamshad, Muzammal Naseer, Karthik Nandakumar, Fahad Shahbaz Khan, Salman Khan

备注：: Accepted at The 5th Workshop of Adversarial Machine Learning on Computer Vision (CVPRW 2025)

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文全面评估了视觉状态空间模型（VSSMs）在遮挡、图像结构、常见损坏和对抗攻击等多种扰动下的鲁棒性，并与Transformer和CNN等架构进行比较，揭示了其优势和局限性。

摘要：

视觉状态空间模型（VSSMs）是一种结合了循环神经网络和潜变量模型优势的新型架构，通过有效捕捉长程依赖和建模复杂视觉动态，在视觉感知任务中表现出色。然而，它们在自然和对抗扰动下的鲁棒性仍然是一个关键问题。在这项工作中，我们全面评估了VSSMs在各种扰动场景下的鲁棒性，包括遮挡、图像结构、常见损坏和对抗攻击，并将其性能与Transformer和卷积神经网络等成熟架构进行比较。此外，我们研究了VSSMs在复杂视觉场景中针对物体-背景组合变化的鲁棒性，使用了专门设计用于测试模型性能的复杂基准。我们还使用模拟真实场景的损坏数据集评估了它们在目标检测和分割任务上的鲁棒性。为了更深入地理解VSSMs的对抗鲁棒性，我们进行了基于频率的对抗攻击分析，评估了它们对低频和高频扰动的性能。我们的发现突出了VSSMs在处理复杂视觉损坏方面的优势和局限性，为未来研究提供了宝贵的见解。我们的代码和模型将在 https://github.com/HashmatShadab/MambaRobustness 提供。

英文摘要：

Vision State Space Models (VSSMs), a novel architecture that combines the strengths of recurrent neural networks and latent variable models, have demonstrated remarkable performance in visual perception tasks by efficiently capturing long-range dependencies and modeling complex visual dynamics. However, their robustness under natural and adversarial perturbations remains a critical concern. In this work, we present a comprehensive evaluation of VSSMs' robustness under various perturbation scenarios, including occlusions, image structure, common corruptions, and adversarial attacks, and compare their performance to well-established architectures such as transformers and Convolutional Neural Networks. Furthermore, we investigate the resilience of VSSMs to object-background compositional changes on sophisticated benchmarks designed to test model performance in complex visual scenes. We also assess their robustness on object detection and segmentation tasks using corrupted datasets that mimic real-world scenarios. To gain a deeper understanding of VSSMs' adversarial robustness, we conduct a frequency-based analysis of adversarial attacks, evaluating their performance against low-frequency and high-frequency perturbations. Our findings highlight the strengths and limitations of VSSMs in handling complex visual corruptions, offering valuable insights for future research. Our code and models will be available at https://github.com/HashmatShadab/MambaRobustness.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。