Deep neural network (DNN) models have proven to be vulnerable to adversarial digital and physical attacks. In this paper, we propose a novel attack- and dataset-agnostic and real-time detector for both types of adversarial inputs to DNN-based perception systems. In particular, the proposed detector relies on the observation that adversarial images are sensitive to certain label-invariant transformations. Specifically, to determine if an image has been adversarially manipulated, the proposed detector checks if the output of the target classifier on a given input image changes significantly after feeding it a transformed version of the image under investigation. Moreover, we show that the proposed detector is computationally-light both at runtime and design-time which makes it suitable for real-time applications that may also involve large-scale image domains. To highlight this, we demonstrate the efficiency of the proposed detector on ImageNet, a task that is computationally challenging for the majority of relevant defenses, and on physically attacked traffic signs that may be encountered in real-time autonomy applications. Finally, we propose the first adversarial dataset, called AdvNet that includes both clean and physical traffic sign images. Our extensive comparative experiments on the MNIST, CIFAR10, ImageNet, and AdvNet datasets show that VisionGuard outperforms existing defenses in terms of scalability and detection performance. We have also evaluated the proposed detector on field test data obtained on a moving vehicle equipped with a perception-based DNN being under attack.
DrivIng: A Large-Scale Multimodal Driving Dataset with Full Digital Twin Integration
Department of Computer Science and AImotion Bavaria, Technische Hochschule Ingolstadt（计算机科学系和AImotion巴伐利亚，因斯布鲁克大学）；School of Computation, Information and Technology, Technical University of Munich（计算、信息与技术学院，慕尼黑技术大学）
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
Recent advancements in Vehicle-to-Everything communication technology have enabled autonomous vehicles to share sensory information to obtain better perception performance. With the rapid growth of autonomous vehicles and intelligent infrastructure, the V2X perception systems will soon be deployed at scale, which raises a safety-critical question: \textit{how can we evaluate and improve its performance under challenging traffic scenarios before the real-world deployment?} Collecting diverse large-scale real-world test scenes seems to be the most straightforward solution, but it is expensive an…
Adversarial examples reveal the vulnerability and unexplained nature of neural networks. Studying the defense of adversarial examples is of considerable practical importance. Most adversarial examples that misclassify networks are often undetectable by humans. In this paper, we propose a defense model to train the classifier into a human-perception classification model with shape preference. The proposed model comprising a texture transfer network (TTN) and an auxiliary defense generative adversarial networks (GAN) is called Human-perception Auxiliary Defense GAN (HAD-GAN). The TTN is used to …
In this study, we introduce a measure for machine perception, inspired by the concept of Just Noticeable Difference (JND) of human perception. Based on this measure, we suggest an adversarial image generation algorithm, which iteratively distorts an image by an additive noise until the model detects the change in the image by outputting a false label. The noise added to the original image is defined as the gradient of the cost function of the model. A novel cost function is defined to explicitly minimize the amount of perturbation applied to the input image while enforcing the perceptual simil…
Recent works have shown that neural networks are vulnerable to carefully crafted adversarial examples (AE). By adding small perturbations to input images, AEs are able to make the victim model predicts incorrect outputs. Several research work in adversarial machine learning started to focus on the detection of AEs in autonomous driving. However, the existing studies either use preliminary assumption on outputs of detections or ignore the tracking system in the perception pipeline. In this paper, we firstly propose a novel distance metric for practical autonomous driving object detection output…
University of Michigan（密歇根大学）；Brown University（布朗大学）
Recent advancements have led to a proliferation of machine learning systems used to assist humans in a wide range of tasks. However, we are still far from accurate, reliable, and resource-efficient operations of these systems. For robot perception, convolutional neural networks (CNNs) for object detection and pose estimation are recently coming into widespread use. However, neural networks are known to suffer overfitting during training process and are less robust within unseen conditions, which are especially vulnerable to adversarial scenarios. In this work, we propose Generative Robust Infe…
Technische Universität Braunschweig（布伦瑞克工业大学）；Valeo DAR（法雷奥驾驶辅助研发部）；Valeo Vision Systems（法雷奥视觉系统公司）
While deep neural networks (DNNs) achieve impressive performance on environment perception tasks, their sensitivity to adversarial perturbations limits their use in practical applications. In this paper, we (i) propose a novel adversarial perturbation detection scheme based on multi-task perception of complex vision tasks (i.e., depth estimation and semantic segmentation). Specifically, adversarial perturbations are detected by inconsistencies between extracted edges of the input image, the depth output, and the segmentation output. To further improve this technique, we (ii) develop a novel ed…
An important goal in human-robot-interaction (HRI) is for machines to achieve a close to human level of face perception. One of the important differences between machine learning and human intelligence is the lack of compositionality. This paper introduces a new scheme to enable generative adversarial networks to learn the distribution of face images composed of smaller parts. This results in a more flexible machine face perception and easier generalization to outside training examples. We demonstrate that this model is able to produce realistic high-quality face images by generating and pieci…
Failure Cases Are Better Learned But Boundary Says Sorry: Facilitating Smooth Perception Change for Accuracy-Robustness Trade-Off in Adversarial Training
The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））
Despite their excellent performance, state-of-the-art computer vision models often fail when they encounter adversarial examples. Video perception models tend to be more fragile under attacks, because the adversary has more places to manipulate in high-dimensional data. In this paper, we find one reason for video models' vulnerability is that they fail to perceive the correct motion under adversarial perturbations. Inspired by the extensive evidence that motion is a key factor for the human visual system, we propose to correct what the model sees by restoring the perceived motion information. …
Sun Yat-sen University（中山大学）；The Chinese University of Hong Kong（香港中文大学）；Royal Brompton Hospital（皇家布朗普顿医院）；Imperial College London（伦敦帝国理工学院）
The long acquisition time has limited the accessibility of magnetic resonance imaging (MRI) because it leads to patient discomfort and motion artifacts. Although several MRI techniques have been proposed to reduce the acquisition time, compressed sensing in magnetic resonance imaging (CS-MRI) enables fast acquisition without compromising SNR and resolution. However, existing CS-MRI methods suffer from the challenge of aliasing artifacts. This challenge results in the noise-like textures and missing the fine details, thus leading to unsatisfactory reconstruction performance. To tackle this chal…
Deep neural networks are vulnerable to adversarial attacks. White-box adversarial attacks can fool neural networks with small adversarial perturbations, especially for large size images. However, keeping successful adversarial perturbations imperceptible is especially challenging for transfer-based black-box adversarial attacks. Often such adversarial examples can be easily spotted due to their unpleasantly poor visual qualities, which compromises the threat of adversarial attacks in practice. In this study, to improve the image quality of black-box adversarial examples perceptually, we propos…
Valeo R&D Egypt（法雷奥研发埃及）；Valeo DAR Germany（法雷奥DAR德国）；Technische Universität Ilmenau（伊尔梅瑙工业大学）；Valeo Ireland（法雷奥爱尔兰）
Deep neural networks (DNNs) have accomplished impressive success in various applications, including autonomous driving perception tasks, in recent years. On the other hand, current deep neural networks are easily fooled by adversarial attacks. This vulnerability raises significant concerns, particularly in safety-critical applications. As a result, research into attacking and defending DNNs has gained much coverage. In this work, detailed adversarial attacks are applied on a diverse multi-task visual perception deep network across distance estimation, semantic segmentation, motion detection, a…
Rain often poses inevitable threats to deep neural network (DNN) based perception systems, and a comprehensive investigation of the potential risks of the rain to DNNs is of great importance. However, it is rather difficult to collect or synthesize rainy images that can represent all rain situations that would possibly occur in the real world. To this end, in this paper, we start from a new perspective and propose to combine two totally different studies, i.e., rainy image synthesis and adversarial attack. We first present an adversarial rain attack, with which we could simulate various rain s…
Histopathology relies on the analysis of microscopic tissue images to diagnose disease. A crucial part of tissue preparation is staining whereby a dye is used to make the salient tissue components more distinguishable. However, differences in laboratory protocols and scanning devices result in significant confounding appearance variation in the corresponding images. This variation increases both human error and the inter-rater variability, as well as hinders the performance of automatic or semi-automatic methods. In the present paper we introduce an unsupervised adversarial network to translat…
Waabi；University of Toronto（多伦多大学）；UIUC（伊利诺伊大学厄巴纳-香槟分校）；Cornell University（康奈尔大学）
Modern self-driving perception systems have been shown to improve upon processing complementary inputs such as LiDAR with images. In isolation, 2D images have been found to be extremely vulnerable to adversarial attacks. Yet, there have been limited studies on the adversarial robustness of multi-modal models that fuse LiDAR features with image features. Furthermore, existing works do not consider physically realizable perturbations that are consistent across the input modalities. In this paper, we showcase practical susceptibilities of multi-sensor detection by placing an adversarial object on…
TUV SUD Singapore（新加坡TUV南德意志集团）；Nanyang Technological University（南洋理工大学）
Autonomous vehicles (AV) have progressed rapidly with the advancements in computer vision algorithms. The deep convolutional neural network as the main contributor to this advancement has boosted the classification accuracy dramatically. However, the discovery of adversarial examples reveals the generalization gap between dataset and the real world. Furthermore, affine transformations may also confuse computer vision based object detectors. The degradation of the perception system is undesirable for safety critical systems such as autonomous vehicles. In this paper, a deep learning system is p…
The limited dynamic range of commercial compact camera sensors results in an inaccurate representation of scenes with varying illumination conditions, adversely affecting image quality and subsequently limiting the performance of underlying image processing algorithms. Current state-of-the-art (SoTA) convolutional neural networks (CNN) are developed as post-processing techniques to independently recover under-/over-exposed images. However, when applied to images containing real-world degradations such as glare, high-beam, color bleeding with varying noise intensity, these algorithms amplify th…
New York University（纽约大学）；University of Chinese Academy of Sciences（中国科学院大学）；Alibaba Group（阿里巴巴集团）
LiDAR point clouds collected from a moving vehicle are functions of its trajectories, because the sensor motion needs to be compensated to avoid distortions. When autonomous vehicles are sending LiDAR point clouds to deep networks for perception and planning, could the motion compensation consequently become a wide-open backdoor in those networks, due to both the adversarial vulnerability of deep learning and GPS-based vehicle trajectory estimation that is susceptible to wireless spoofing? We demonstrate such possibilities for the first time: instead of directly attacking point cloud coordinat…
We present a novel generalized zero-shot algorithm to recognize perceived emotions from gestures. Our task is to map gestures to novel emotion categories not encountered in training. We introduce an adversarial, autoencoder-based representation learning that correlates 3D motion-captured gesture sequence with the vectorized representation of the natural-language perceived emotion terms using word2vec embeddings. The language-semantic embedding provides a representation of the emotion label space, and we leverage this underlying distribution to map the gesture-sequences to the appropriate categ…
Salzburg University of Applied Sciences（萨尔茨堡应用科学大学）
Image analysis in the field of digital pathology has recently gained increased popularity. The use of high-quality whole slide scanners enables the fast acquisition of large amounts of image data, showing extensive context and microscopic detail at the same time. Simultaneously, novel machine learning algorithms have boosted the performance of image analysis approaches. In this paper, we focus on a particularly powerful class of architectures, called Generative Adversarial Networks (GANs), applied to histological image data. Besides improving performance, GANs also enable application scenarios…
AdvGPS: Adversarial GPS for Multi-Agent Perception Attack
Cleveland State University（克利夫兰州立大学）；Xi’an Jiaotong University（西安交通大学）；New York University（纽约大学）；Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局）；CFAR（计算与数据科学中心）；IHPC（高性能计算研究所）；DSO National Laboratories（新加坡国防科技局国家实验室）
Wuhan University of Technology（武汉理工大学）；Hubei Key Laboratory of Broadband Wireless Communication and Sensor Networks（湖北省宽带无线通信与传感器网络重点实验室）；School of Civil Engineering & Architecture（土木工程与建筑学院）；School of Information Engineering（信息工程学院）；State Key Laboratory of Magnetic Resonance and Atomic Molecular Phys（磁共振与原子分子物理国家重点实验室）
The presence of residual and dense neural networks which greatly promotes the development of image Super-Resolution(SR) have witnessed a lot of impressive results. Depending on our observation, although more layers and connections could always improve performance, the increase of model parameters is not conducive to launch application of SR algorithms. Furthermore, algorithms supervised by L1/L2 loss can achieve considerable performance on traditional metrics such as PSNR and SSIM, yet resulting in blurry and over-smoothed outputs without sufficient high-frequency details, namely low perceptua…
CARNet: Collaborative Adversarial Resilience for Robust Underwater Image Enhancement and Perception
School of Software Technology, Dalian University of Technology（大连理工大学软件学院）；College of Information Science and Technology, Dalian Maritime University（大连海事大学信息科学技术学院）；School of Mechanical Engineering, Dalian University of Technology（大连理工大学机械工程学院）
All Vehicles Can Lie: Efficient Adversarial Defense in Fully Untrusted-Vehicle Collaborative Perception via Pseudo-Random Bayesian Inference
School of Cyber Science and Engineering, Wuhan University（武汉大学网络科学与工程学院）；Cyberspace Institute of Advanced Technology, Guangzhou University（广州大学先进技术网络研究院）
Reinforced Embodied Active Defense: Exploiting Adaptive Interaction for Robust Visual Perception in Adversarial 3D Environments
Dept. of Comp. Sci. & Tech.（计算机科学与技术系）；Institute for AI（人工智能研究院）；BNRist Center（BNRist中心）；THBI Lab（THBI实验室）；Tsinghua-Bosch Joint Center for ML（清华大学-博世联合机器学习中心）；Tsinghua University（清华大学）
Zhejiang University of Science and Technology（浙江科技大学）；Hangzhou Hikvision Digital Technology Co., Ltd.（杭州海康威视数字技术股份有限公司）；University of Electronic Science and Technology of China（电子科技大学）；Guangzhou University（广州大学）；Zhejiang University（浙江大学）；CY Cergy Paris University（CY塞尔吉巴黎大学）
The vulnerability of deep neural networks (DNNs) to adversarial examples has attracted more attention. Many algorithms have been proposed to craft powerful adversarial examples. However, most of these algorithms modified the global or local region of pixels without taking network explanations into account. Hence, the perturbations are redundant, which are easily detected by human eyes. In this paper, we propose a novel method to generate local region perturbations. The main idea is to find a contributing feature region (CFR) of an image by simulating the human attention mechanism and then add …
Robustifying 3D Perception via Least-Squares Graphs for Multi-Agent Object Tracking
Industrial Systems Institute, Athena Research Center, Patras Science Park, Greece（希腊雅典研究中心工业系统研究所）；AviSense.AI, Patras Science Park, Greece（希腊帕特拉斯科学公园AviSense.AI）；Dpt. of Informatics & Telecom., University of Ioannina, Arta, Greece（希腊伊奥安妮大学信息与电信系）
Visa Research（Visa研究院）；Michigan State University（密歇根州立大学）
Deep learning-based systems have been shown to be vulnerable to adversarial attacks in both digital and physical domains. While feasible, digital attacks have limited applicability in attacking deployed systems, including face recognition systems, where an adversary typically has access to the input and not the transmission channel. In such setting, physical attacks that directly provide a malicious input through the input channel pose a bigger threat. We investigate the feasibility of conducting real-time physical attacks on face recognition systems using adversarial light projections. A setu…
Perception plays a pivotal role in autonomous driving systems, which utilizes onboard sensors like cameras and LiDARs (Light Detection and Ranging) to assess surroundings. Recent studies have demonstrated that LiDAR-based perception is vulnerable to spoofing attacks, in which adversaries spoof a fake vehicle in front of a victim self-driving car by strategically transmitting laser signals to the victim's LiDAR sensor. However, existing attacks suffer from effectiveness and generality limitations. In this work, we perform the first study to explore the general vulnerability of current LiDAR-bas…
University of British Columbia（英属哥伦比亚大学）；Department of Statistics, University of British Columbia（英属哥伦比亚大学统计系）；Department of Electrical and Computer Engineering, University of British Columbia（英属哥伦比亚大学电气与计算机工程系）；School of Information and Communications Engineering, Xi’an Jiaotong University（西安交通大学信息与通信工程学院）
Recently, generative adversarial networks (GANs) can generate photo-realistic fake facial images which are perceptually indistinguishable from real face photos, promoting research on fake face detection. Though fake face forensics can achieve high detection accuracy, their anti-forensic counterparts are less investigated. Here we explore more \textit{imperceptible} and \textit{transferable} anti-forensics for fake face imagery detection based on adversarial attacks. Since facial and background regions are often smooth, even small perturbation could cause noticeable perceptual impairment in fak…
In this paper, we propose a novel virtual reality image quality assessment (VR IQA) with adversarial learning for omnidirectional images. To take into account the characteristics of the omnidirectional image, we devise deep networks including novel quality score predictor and human perception guider. The proposed quality score predictor automatically predicts the quality score of distorted image using the latent spatial and position feature. The proposed human perception guider criticizes the predicted quality score of the predictor with the human perceptual score using adversarial learning. F…
The availability and easy access to digital communication increase the risk of copyrighted material piracy. In order to detect illegal use or distribution of data, digital watermarking has been proposed as a suitable tool. It protects the copyright of digital content by embedding imperceptible information into the data in the presence of an adversary. The goal of the adversary is to remove the copyrighted content of the data. Therefore, an efficient watermarking framework must be robust to multiple image-processing operations known as attacks that can alter embedded copyright information. Anot…
Beyond Domain Randomization: Event-Inspired Perception for Visually Robust Adversarial Imitation from Videos
Mercedes-Benz AG（梅赛德斯-奔驰集团）；Technical University of Munich（慕尼黑技术大学）；Purdue University（普渡大学）；Space Robotics Research Group, SnT, University of Luxembourg（卢森堡大学空间机器人研究组）；Torc Robotics（Torc机器人公司）；Princeton University（普林斯顿大学）
Unveiling Hidden Vulnerabilities in Digital Human Generation via Adversarial Attacks
College of Cyber Security, Jinan University（济南大学网络安全学院）；Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University（北京未来区块链与隐私计算先进创新中心，北航人工智能学院）；Hangzhou International Innovation Institute, Beihang University（杭州国际创新研究院，北航）；Department of Electrical and Computer Engineering, National University of Singapore（新加坡国立大学电子与计算机工程系）；Department of Electrical and Computer Science, University of Pittsburgh（匹兹堡大学电子与计算机科学系）；State Key Laboratory of Multimodal Artificial Intelligence, Institute of Automation, Chinese Academy of Science（中国科学院多模态人工智能国家重点实验室，自动化研究所）；Institute of Automation, Chinese Academy of Science（中国科学院自动化研究所）；University College London（伦敦大学学院）；Mingdu Tech（明都科技）；Xreal；Beijing Academy of Blockchain and Edge Computing（北京区块链与边缘计算研究院）
