Modern autonomous driving systems rely heavily on deep learning models to process point cloud sensory data; meanwhile, deep models have been shown to be susceptible to adversarial attacks with visually imperceptible perturbations. Despite the fact that this poses a security concern for the self-driving industry, there has been very little exploration in terms of 3D perception, as most adversarial attacks have only been applied to 2D flat images. In this paper, we address this issue and present a method to generate universal 3D adversarial objects to fool LiDAR detectors. In particular, we demonstrate that placing an adversarial object on the rooftop of any target vehicle to hide the vehicle entirely from LiDAR detectors with a success rate of 80%. We report attack results on a suite of detectors using various input representation of point clouds. We also conduct a pilot study on adversarial defense using data augmentation. This is one step closer towards safer self-driving under unseen conditions from limited training data.
PBCAT: Patch-based composite adversarial training against physically realizable attacks on object detection
Department of Computer Science and Technology, BNRist, IDG/McGovern Institute for Brain Research, THBI, Tsinghua University（清华大学计算机科学与技术系）；University of Science and Technology Beijing（北京科技大学）；Huawei Technologies（华为技术有限公司）；Chinese Institute for Brain Research (CIBR)（中国脑科学研究院）
Washington University in St. Louis（圣路易斯华盛顿大学）；Department of Computer Science and Engineering（计算机科学与工程系）
We study the problem of defending deep neural network approaches for image classification from physically realizable attacks. First, we demonstrate that the two most scalable and effective methods for learning robust models, adversarial training with PGD attacks and randomized smoothing, exhibit very limited effectiveness against three of the highest profile physical attacks. Next, we propose a new abstract adversarial model, rectangular occlusion attacks, in which an adversary places a small adversarially crafted rectangle in an image, and develop two approaches for efficiently computing the …
Towards Physically Realizable Adversarial Attenuation Patch against SAR Object Detection
Key Laboratory for Information Science of Electromagnetic Waves (MoE) School of Information Science and Technology, Fudan University（电磁波信息科学重点实验室（MoE）复旦大学信息科学与技术学院）
Consistent Valid Physically-Realizable Adversarial Attack against Crowd-flow Prediction Models
Information Technology University (ITU)（信息技术大学）；Qatar Mobility Innovations Center (QMIC)（卡塔尔移动创新中心）；Hamad Bin Khalifa University (HBKU)（哈马德·本·哈利法大学）；Qatar University（卡塔尔大学）
Bosch Center for Artificial Intelligence（博世人工智能中心）；Carnegie Mellon University（卡内基梅隆大学）
Recent work has documented the susceptibility of deep learning systems to adversarial examples, but most such attacks directly manipulate the digital input to a classifier. Although a smaller line of work considers physical adversarial attacks, in all cases these involve manipulating the object of interest, e.g., putting a physical sticker on an object to misclassify it, or manufacturing an object specifically intended to be misclassified. In this work, we consider an alternative question: is it possible to fool deep classifiers, over all perceived objects of a certain type, by physically mani…
Waabi；University of Toronto（多伦多大学）；UIUC（伊利诺伊大学厄巴纳-香槟分校）；Cornell University（康奈尔大学）
Modern self-driving perception systems have been shown to improve upon processing complementary inputs such as LiDAR with images. In isolation, 2D images have been found to be extremely vulnerable to adversarial attacks. Yet, there have been limited studies on the adversarial robustness of multi-modal models that fuse LiDAR features with image features. Furthermore, existing works do not consider physically realizable perturbations that are consistent across the input modalities. In this paper, we showcase practical susceptibilities of multi-sensor detection by placing an adversarial object on…
ETH Zürich（苏黎世联邦理工学院）；MPI for Informatics（马克斯·普朗克信息学研究所）；KU Leuven（鲁汶大学）
This work addresses the challenging task of LiDAR-based 3D object detection in foggy weather. Collecting and annotating data in such a scenario is very time, labor and cost intensive. In this paper, we tackle this problem by simulating physically accurate fog into clear-weather scenes, so that the abundant existing real datasets captured in clear weather can be repurposed for our task. Our contributions are twofold: 1) We develop a physically valid fog simulation method that is applicable to any LiDAR dataset. This unleashes the acquisition of large-scale foggy training data at no extra cost. …
University of British Columbia（不列颠哥伦比亚大学）；Department of Electrical and Computer Engineering, University of British Columbia（不列颠哥伦比亚大学电气与计算机工程系）
Most autonomous vehicles (AVs) rely on LiDAR and RGB camera sensors for perception. Using these point cloud and image data, perception models based on deep neural nets (DNNs) have achieved state-of-the-art performance in 3D detection. The vulnerability of DNNs to adversarial attacks has been heavily investigated in the RGB image domain and more recently in the point cloud domain, but rarely in both domains simultaneously. Multi-modal perception systems used in AVs can be divided into two broad types: cascaded models which use each modality independently, and fusion models which learn from diff…
ETH Zürich（苏黎世联邦理工学院）；Princeton University（普林斯顿大学）；MPI for Informatics（马克斯·普朗克信息学研究所）；KU Leuven（鲁汶大学）
3D object detection is a central task for applications such as autonomous driving, in which the system needs to localize and classify surrounding traffic agents, even in the presence of adverse weather. In this paper, we address the problem of LiDAR-based 3D object detection under snowfall. Due to the difficulty of collecting and annotating training data in this setting, we propose a physically based method to simulate the effect of snowfall on real clear-weather LiDAR point clouds. Our method samples snow particles in 2D space for each LiDAR line and uses the induced geometry to modify the me…
We propose a universal and physically realizable adversarial attack on a cascaded multi-modal deep learning network (DNN), in the context of self-driving cars. DNNs have achieved high performance in 3D object detection, but they are known to be vulnerable to adversarial attacks. These attacks have been heavily investigated in the RGB image domain and more recently in the point cloud domain, but rarely in both domains simultaneously - a gap to be filled in this paper. We use a single 3D mesh and differentiable rendering to explore how perturbing the mesh's geometry and texture can reduce the ro…
SRI International（斯坦福国际研究院）；University of Central Florida（中佛罗里达大学）；University of Wisconsin（威斯康星大学）；U.S. Army Research Laboratory（美国陆军研究实验室）
Attribution methods have been developed to explain the decision of a machine learning model on a given input. We use the Integrated Gradient method for finding attributions to define the causal neighborhood of an input by incrementally masking high attribution features. We study the robustness of machine learning models on benign and adversarial inputs in this neighborhood. Our study indicates that benign inputs are robust to the masking of high attribution features but adversarial inputs generated by the state-of-the-art adversarial attack methods such as DeepFool, FGSM, CW and PGD, are not r…
OBJVanish: Physically Realizable Text-to-3D Adv. Generation of LiDAR-Invisible Objects
CFAR, A*STAR（CFAR、A*STAR）；Chang’an University（长安大学）；Hefei University of Technology（合肥工业大学）；Zhongguancun Laboratory（中关村实验室）；Tianjin University（天津大学）；University of Warwick（沃里克大学）
OmniLiDAR: A Unified Diffusion Framework for Multi-Domain 3D LiDAR Generation
College of Computer Science and Artificial Intelligence, Fudan University（复旦大学计算机科学与人工智能学院）；School of Computing, Department of Computer Science, National University of Singapore（新加坡国立大学计算机学院）；College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics（南京航空航天大学计算机科学与技术学院）；Technical University of Munich（慕尼黑技术大学）；Nanjing University of Science and Technology（南京理工大学）；Shanghai AI Laboratory（上海人工智能实验室）；University of Sydney（悉尼大学）；The Chinese University of Hong Kong, Hong Kong SAR（香港中文大学（深圳））
Adv3D: Generating 3D Adversarial Examples for 3D Object Detection in Driving Scenarios with NeRF
Artificial Intelligence Thrust, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）人工智能学科）；Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（香港科技大学计算机科学与工程学系）
本文提出 Adv3D，首次将对抗样本建模为 NeRF，并通过基元感知采样与语义引导正则化生成可物理实现的 3D 伪装纹理补丁，攻击驾驶场景中的 3D 检测器。
Person tracking using computer vision techniques has wide ranging applications such as autonomous driving, home security and sports analytics. However, the growing threat of adversarial attacks raises serious concerns regarding the security and reliability of such techniques. In this work, we study the impact of multi-task learning (MTL) on the adversarial robustness of the widely used SiamRPN tracker, in the context of person tracking. Specifically, we investigate the effect of jointly learning with semantically analogous tasks of person tracking and human keypoint detection. We conduct exten…
The majority of adversarial machine learning research focuses on additive attacks, which add adversarial perturbation to input data. On the other hand, unlike image recognition problems, only a handful of attack approaches have been explored in the video domain. In this paper, we propose a novel attack method against video recognition models, Multiplicative Adversarial Videos (MultAV), which imposes perturbation on video data by multiplication. MultAV has different noise distributions to the additive counterparts and thus challenges the defense methods tailored to resisting additive adversaria…
University of Michigan（密歇根大学）；NVIDIA Research（英伟达研究院）；Arizona State University（亚利桑那州立大学）；Inceptio（嬴彻科技）；Baidu Research（百度研究院）；National Engineering Laboratory of Deep Learning Technology and Application（深度学习技术及应用国家工程实验室）；University of California, Irvine（加州大学欧文分校）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
In Autonomous Driving (AD) systems, perception is both security and safety critical. Despite various prior studies on its security issues, all of them only consider attacks on camera- or LiDAR-based AD perception alone. However, production AD systems today predominantly adopt a Multi-Sensor Fusion (MSF) based design, which in principle can be more robust against these attacks under the assumption that not all fusion sources are (or can be) attacked at the same time. In this paper, we present the first study of security issues of MSF-based perception in AD systems. We directly challenge the bas…
Detectors Learn the Wrong Thing: Shortcut-Resistant Adversarial Training Against Physically Realizable Attacks
School of Transportation Science and Engineering, Beihang University（北京航空航天大学交通科学与工程学院）；State Key Lab of Intelligent Transportation System（智能交通系统国家重点实验室）；Zhongguancun Laboratory（中关村实验室）；School of Systems Science and Engineering, Sun Yat-Sen University（中山大学系统科学与工程学院）；Shandong Sino-Aisa Tire Proving Ground Co.,Ltd.（山东中亚洲轮胎试验场有限公司）
A First Physical-World Trajectory Prediction Attack via LiDAR-induced Deceptions in Autonomous Driving
City University of Hong Kong（香港城市大学）；State University of New York at Buffalo（纽约州立大学布法罗分校）；Delft University of Technology（代尔夫特理工大学）；Nanyang Technological University（南洋理工大学）；Hon Hai Research Institute（鸿海研究院）；Feng Chia University（逢甲大学）
LiDAR data is becoming increasingly essential with the rise of autonomous vehicles. Its ability to provide 360deg horizontal field of view of point cloud, equips self-driving vehicles with enhanced situational awareness capabilities. While synthetic LiDAR data generation pipelines provide a good solution to advance the machine learning research on LiDAR, they do suffer from a major shortcoming, which is rendering time. Physically accurate LiDAR simulators (e.g. Blensor) are computationally expensive with an average rendering time of 14-60 seconds per frame for urban scenes. This is often compe…
The Johns Hopkins University Applied Physics Laboratory（约翰霍普金斯大学应用物理实验室）
Perturbation-based attacks, while not physically realizable, have been the main emphasis of adversarial machine learning (ML) research. Patch-based attacks by contrast are physically realizable, yet most work has focused on 2D domain with recent forays into 3D. Characterizing the robustness properties of patch attacks and their invariance to 3D pose is important, yet not fully elucidated, and is the focus of this paper. To this end, several contributions are made here: A) we develop a new metric called mean Attack Success over Transformations (mAST) to evaluate patch attack robustness and inva…
The University of Adelaide（阿德莱德大学）；Rutgers University（罗格斯大学）
Deep neural networks are vulnerable to attacks from adversarial inputs and, more recently, Trojans to misguide or hijack the model's decision. We expose the existence of an intriguing class of spatially bounded, physically realizable, adversarial examples -- Universal NaTuralistic adversarial paTches -- we call TnTs, by exploring the superset of the spatially bounded adversarial example space and the natural input space within generative adversarial networks. Now, an adversary can arm themselves with a patch that is naturalistic, less malicious-looking, physically realizable, highly effective …
CAAP: Capture-Aware Adversarial Patch Attacks on Palmprint Recognition Models
Institute of Data Science, National University of Singapore（新加坡国立大学数据科学研究所）；Centre for Frontier AI Research (CFAR), A*STAR（A*STAR前沿人工智能研究中心）；School of Cyber Science and Engineering, Wuhan University（武汉大学网络空间安全学院）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；School of Engineering, Yunnan University（云南大学工程学院）
