SAM的鲁棒性：在损坏及更多场景下的分割一切
Robustness of SAM: Segment Anything Under Corruptions and Beyond
- Kyung Hee University（庆熙大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本研究首次全面评估SAM在风格变化、15种常见损坏、局部遮挡及对抗攻击下的鲁棒性，揭示其实际部署中的性能局限。
AI中文摘要：
分割一切模型（SAM），顾名思义，声称能够分割出任何物体，并在提示引导下展现出令人印象深刻的零样本迁移性能。然而，目前缺乏关于其在各种损坏条件下鲁棒性的全面评估。理解SAM在不同损坏场景下的鲁棒性对其实际部署至关重要。先前的工作表明，SAM偏向于纹理（风格）而非形状，受此启发，我们首先研究其对风格迁移（一种合成损坏）的鲁棒性。接着，将合成损坏的影响解释为风格变化，我们对其在15种常见损坏类型下的鲁棒性进行了全面评估。这些损坏主要分为数字、噪声、天气和模糊等类别，在每个损坏类别中，我们探索了5个严重程度级别以模拟真实世界的损坏场景。在损坏之外，我们进一步评估了SAM对局部遮挡和局部对抗补丁攻击的鲁棒性。据我们所知，我们的工作是首个评估SAM在风格变化、局部遮挡和局部对抗补丁攻击下鲁棒性的研究。鉴于肉眼可见的补丁攻击容易被检测，我们进一步评估了其对肉眼不可见的全局对抗攻击的鲁棒性。总体而言，这项工作对SAM的鲁棒性进行了全面的实证研究，评估了其在各种损坏下的性能，并将评估扩展到局部遮挡、局部对抗补丁攻击和全局对抗攻击等关键方面。这些评估为SAM在实际应用中应对现实世界挑战的实用性和有效性提供了宝贵见解。
英文摘要：
Segment anything model (SAM), as the name suggests, is claimed to be capable of cutting out any object and demonstrates impressive zero-shot transfer performance with the guidance of prompts. However, there is currently a lack of comprehensive evaluation regarding its robustness under various corruptions. Understanding the robustness of SAM across different corruption scenarios is crucial for its real-world deployment. Prior works show that SAM is biased towards texture (style) rather than shape, motivated by which we start by investigating its robustness against style transfer, which is synthetic corruption. Following by interpreting the effects of synthetic corruption as style changes, we proceed to conduct a comprehensive evaluation for its robustness against 15 types of common corruption. These corruptions mainly fall into categories such as digital, noise, weather, and blur, and within each corruption category, we explore 5 severity levels to simulate real-world corruption scenarios. Beyond the corruptions, we further assess the robustness of SAM against local occlusion and local adversarial patch attacks. To the best of our knowledge, our work is the first of its kind to evaluate the robustness of SAM under style change, local occlusion, and local adversarial patch attacks. Given that patch attacks visible to human eyes are easily detectable, we further assess its robustness against global adversarial attacks that are imperceptible to human eyes. Overall, this work provides a comprehensive empirical study of the robustness of SAM, evaluating its performance under various corruptions and extending the assessment to critical aspects such as local occlusion, local adversarial patch attacks, and global adversarial attacks. These evaluations yield valuable insights into the practical applicability and effectiveness of SAM in addressing real-world challenges.
