深度神经网络对对抗性补丁的脆弱性促使了许多防御策略的提出，以提升模型鲁棒性。然而，主流防御方法依赖于单次观测或预先建立的对抗者信息来对抗对抗性补丁，往往无法应对未见过的或自适应的对抗攻击，并且在动态3D环境中容易表现出不令人满意的性能。受主动人类感知和循环反馈机制的启发，我们提出了具身主动防御（Embodied Active Defense，EAD），这是一种主动防御策略，能够主动结合环境信息来应对3D真实世界场景中未对齐的对抗性补丁。为实现这一目标，EAD设计了两个核心循环子模块，即感知模块和策略模块，以实现主动视觉的两个关键功能。这些模型循环处理一系列信念和观测，促进其对目标物体理解的逐步细化，并使其能够制定策略性动作以对抗3D环境中的对抗性补丁。为优化学习效率，我们引入了环境动态的可微近似，并部署了与对抗者策略无关的补丁。大量实验表明，在安全关键任务（例如人脸识别和物体检测）中，EAD通过其动作策略在仅几步内显著增强了对多种补丁的鲁棒性，且不损害标准准确率。此外，由于具有攻击无关特性，EAD对未见过的攻击表现出优异的泛化能力，在一系列未见过的对抗攻击中平均攻击成功率降低了95%。
英文摘要：
The vulnerability of deep neural networks to adversarial patches has motivated numerous defense strategies for boosting model robustness. However, the prevailing defenses depend on single observation or pre-established adversary information to counter adversarial patches, often failing to be confronted with unseen or adaptive adversarial attacks and easily exhibiting unsatisfying performance in dynamic 3D environments. Inspired by active human perception and recurrent feedback mechanisms, we develop Embodied Active Defense (EAD), a proactive defensive strategy that actively contextualizes environmental information to address misaligned adversarial patches in 3D real-world settings. To achieve this, EAD develops two central recurrent sub-modules, i.e., a perception module and a policy module, to implement two critical functions of active vision. These models recurrently process a series of beliefs and observations, facilitating progressive refinement of their comprehension of the target object and enabling the development of strategic actions to counter adversarial patches in 3D environments. To optimize learning efficiency, we incorporate a differentiable approximation of environmental dynamics and deploy patches that are agnostic to the adversary strategies. Extensive experiments demonstrate that EAD substantially enhances robustness against a variety of patches within just a few steps through its action policy in safety-critical tasks (e.g., face recognition and object detection), without compromising standard accuracy. Furthermore, due to the attack-agnostic characteristic, EAD facilitates excellent generalization to unseen attacks, diminishing the averaged attack success rate by 95 percent across a range of unseen adversarial attacks.
Reinforced Embodied Active Defense: Exploiting Adaptive Interaction for Robust Visual Perception in Adversarial 3D Environments
Dept. of Comp. Sci. & Tech.（计算机科学与技术系）；Institute for AI（人工智能研究院）；BNRist Center（BNRist中心）；THBI Lab（THBI实验室）；Tsinghua-Bosch Joint Center for ML（清华大学-博世联合机器学习中心）；Tsinghua University（清华大学）
I Don't Know You, But I Can Catch You: Real-Time Defense against Diverse Adversarial Patches for Object Detectors
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；University of Chinese Academy of Sciences（中国科学院大学）；School of Cyber Security, UCAS（中国科学院大学网络空间安全学院）
Defending Against Malicious Finetuning by Scaling Train-time Adversarial Attacks
Xiongan AI Institute（雄安人工智能研究院）；Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）；Shanghai Qi Zhi Institute（上海期智研究院）
PBCAT: Patch-based composite adversarial training against physically realizable attacks on object detection
Department of Computer Science and Technology, BNRist, IDG/McGovern Institute for Brain Research, THBI, Tsinghua University（清华大学计算机科学与技术系）；University of Science and Technology Beijing（北京科技大学）；Huawei Technologies（华为技术有限公司）；Chinese Institute for Brain Research (CIBR)（中国脑科学研究院）
Bosch Center for Artificial Intelligence（博世人工智能中心）
Recently demonstrated physical-world adversarial attacks have exposed vulnerabilities in perception systems that pose severe risks for safety-critical applications such as autonomous driving. These attacks place adversarial artifacts in the physical world that indirectly cause the addition of a universal patch to inputs of a model that can fool it in a variety of contexts. Adversarial training is the most effective defense against image-dependent adversarial attacks. However, tailoring adversarial training to universal patches is computationally expensive since the optimal universal patch depe…
DIFFender: Diffusion-Based Adversarial Defense against Patch Attacks
Institute of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院）；Tsinghua-Bosch Joint ML Center, Tsinghua University（清华大学-博世联合机器学习中心）；Hangzhou Innovation Institute, Beihang University（北京航空航天大学杭州创新研究院）；Zhongguancun Laboratory（中关村实验室）；RealAI
University of California, Berkeley（加州大学伯克利分校）；University of Michigan（密歇根大学）
Deep learning image classification is vulnerable to adversarial attack, even if the attacker changes just a small patch of the image. We propose a defense against patch attacks based on partially occluding the image around each candidate patch location, so that a few occlusions each completely hide the patch. We demonstrate on CIFAR-10, Fashion MNIST, and MNIST that our defense provides certified security against patch attacks of a certain size.
Bosch Center for Artificial Intelligence, Robert Bosch GmbH（罗伯特·博世有限公司博世人工智能中心）
Adversarial patches pose a realistic threat model for physical world attacks on autonomous systems via their perception component. Autonomous systems in safety-critical domains such as automated driving should thus contain a fail-safe fallback component that combines certifiable robustness against patches with efficient inference while maintaining high performance on clean inputs. We propose BagCert, a novel combination of model architecture and certification procedure that allows efficient certification. We derive a loss that enables end-to-end optimization of certified robustness against pat…
Princeton University（普林斯顿大学）；University of Chicago（芝加哥大学）
Localized adversarial patches aim to induce misclassification in machine learning models by arbitrarily modifying pixels within a restricted region of an image. Such attacks can be realized in the physical world by attaching the adversarial patch to the object to be misclassified, and defending against such attacks is an unsolved/open problem. In this paper, we propose a general defense framework called PatchGuard that can achieve high provable robustness while maintaining high clean accuracy against localized adversarial patches. The cornerstone of PatchGuard involves the use of CNNs with sma…
Qian Xuesen Laboratory of Space Technology（钱学森空间技术实验室）；China Academy of Space Technology（中国空间技术研究院）
The security of object detection systems has attracted increasing attention, especially when facing adversarial patch attacks. Since patch attacks change the pixels in a restricted area on objects, they are easy to implement in the physical world, especially for attacking human detection systems. The existing defenses against patch attacks are mostly applied for image classification problems and have difficulty resisting human detection attacks. Towards this critical issue, we propose an efficient and effective plug-in defense component on the YOLO detection system, which we name Ad-YOLO. The …
Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses
Fudan University（复旦大学）；Shanghai Innovation Institute（上海创新研究院）；City University of Hong Kong（香港城市大学）；Jilin University（吉林大学）；Singapore Management University（新加坡管理大学）；Deakin University（德肯大学）；Tongji University（同济大学）；Nanyang Technological University（南洋理工大学）；Chinese Academy of Sciences（中国科学院）；The University of Melbourne（墨尔本大学）；Johns Hopkins University（约翰霍普金斯大学）
PAD: Patch-Agnostic Defense against Adversarial Patch Attacks
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区网络空间安全学院）
The adversarial patch attack against image classification models aims to inject adversarially crafted pixels within a restricted image region (i.e., a patch) for inducing model misclassification. This attack can be realized in the physical world by printing and attaching the patch to the victim object; thus, it imposes a real-world threat to computer vision systems. To counter this threat, we design PatchCleanser as a certifiably robust defense against adversarial patches. In PatchCleanser, we perform two rounds of pixel masking on the input image to neutralize the effect of the adversarial pa…
Architecture-agnostic Iterative Black-box Certified Defense against Adversarial Patches
East China Normal University（华东师范大学）；Nanyang Technological University（南洋理工大学）；Centre for Frontier AI Research (CFAR), A*STAR（新加坡科技研究局前沿人工智能研究中心（CFAR））；New York University（纽约大学）
Institute of Computing Technology, CAS（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Cambricon Technologies（寒武纪科技）；UC Santa Barbara（加州大学圣巴巴拉分校）；Northeastern University（东北大学）；Drexel University（德雷塞尔大学）
Adversarial patch attacks that craft the pixels in a confined region of the input images show their powerful attack effectiveness in physical environments even with noises or deformations. Existing certified defenses towards adversarial patch attacks work well on small images like MNIST and CIFAR-10 datasets, but achieve very poor certified accuracy on higher-resolution images like ImageNet. It is urgent to design both robust and effective defenses against such a practical and harmful attack in industry-level larger images. In this work, we propose the certified defense methodology that achiev…
Adversarial Semantic and Label Perturbation Attack for Pedestrian Attribute Recognition
School of Artificial Intelligence, Anhui University（安徽大学人工智能学院）；School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）；National University of Defense Technology（国防科学技术大学）；Harbin Institute of Technology（哈尔滨工业大学）；Peng Cheng Laboratory（鹏城实验室）
Apple Inc.（苹果公司）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）
Deep neural networks, although shown to be a successful class of machine learning algorithms, are known to be extremely unstable to adversarial perturbations. Improving the robustness of neural networks against these attacks is important, especially for security-critical applications. To defend against such attacks, we propose dividing the input image into multiple patches, denoising each patch independently, and reconstructing the image, without losing significant image content. We call our method D3. This proposed defense mechanism is non-differentiable which makes it non-trivial for an adve…
The Weizmann Institute of Science（魏茨曼科学研究所）；Weizmann Artificial Intelligence Center (WAIC)（魏茨曼人工智能中心）
Single image generative models perform synthesis and manipulation tasks by capturing the distribution of patches within a single image. The classical (pre Deep Learning) prevailing approaches for these tasks are based on an optimization process that maximizes patch similarity between the input and generated output. Recently, however, Single Image GANs were introduced both as a superior solution for such manipulation tasks, but also for remarkable novel generative tasks. Despite their impressiveness, single image GANs require long training time (usually hours) for each image and each task. They…
CP-uniGuard: A Unified, Probability-Agnostic, and Adaptive Framework for Malicious Agent Detection and Defense in Multi-Agent Embodied Perception Systems
Hong Kong JC STEM Lab of Smart City and Department of Computer Science, City University of Hong Kong（香港JC STEM实验室及城市大学计算机科学系）；School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；Department of Electrical and Electronic Engineering, The University of Hong Kong（香港大学电子与电气工程系）；School of Data Science, Lingnan University（岭南大学数据科学学院）
Patch-based adversarial attacks introduce a perceptible but localized change to the input that induces misclassification. While progress has been made in defending against imperceptible attacks, it remains unclear how patch-based attacks can be resisted. In this work, we study two different approaches for defending against black-box patch attacks. First, we show that adversarial training, which is successful against imperceptible attacks, has limited effectiveness against state-of-the-art location-optimized patch attacks. Second, we find that compositional deep networks, which have part-based …
Jedi: Entropy-based Localization and Removal of Adversarial Patches
Université de Sousse（苏塞大学）；Ecole Nationale d’Ingénieurs de Sousse（苏塞国家工程师学院）；University of California Riverside（加州大学河滨分校）；Queen’s University Belfast（贝尔法斯特女王大学）；CNRS（法国国家科学研究中心）；Université Polytechnique Hauts-de-France（上法兰西综合理工大学）；Université de Jendouba（坚杜巴大学）；Institut National des Technologies et des Sciences du Kef（卡夫国家技术与科学学院）
Adversarial patch attacks mislead neural networks by injecting adversarial pixels within a local region. Patch attacks can be highly effective in a variety of tasks and physically realizable via attachment (e.g. a sticker) to the real-world objects. Despite the diversity in attack patterns, adversarial patches tend to be highly textured and different in appearance from natural images. We exploit this property and present PatchZero, a general defense pipeline against white-box adversarial patches without retraining the downstream classifier or detector. Specifically, our defense detects adversa…
NutVLM: A Self-Adaptive Defense Framework against Full-Dimension Attacks for Vision Language Models in Autonomous Driving
Department of Control Science and Engineering, Harbin Institute of Technology（控制科学与工程系，哈尔滨工业大学）；College of Computing and Data Science, Nanyang Technological University（计算与数据科学学院，南洋理工大学）
An adversarial patch can arbitrarily manipulate image pixels within a restricted region to induce model misclassification. The threat of this localized attack has gained significant attention because the adversary can mount a physically-realizable attack by attaching patches to the victim object. Recent provably robust defenses generally follow the PatchGuard framework by using CNNs with small receptive fields and secure feature aggregation for robust model predictions. In this paper, we extend PatchGuard to PatchGuard++ for provably detecting the adversarial patch attack to boost both provabl…
