With the wide bandwidths in millimeter wave (mmWave) frequency band that results in unprecedented accuracy, mmWave sensing has become vital for many applications, especially in autonomous vehicles (AVs). In addition, mmWave sensing has superior reliability compared to other sensing counterparts such as camera and LiDAR, which is essential for safety-critical driving. Therefore, it is critical to understand the security vulnerabilities and improve the security and reliability of mmWave sensing in AVs. To this end, we perform the end-to-end security analysis of a mmWave-based sensing system in AVs, by designing and implementing practical physical layer attack and defense strategies in a state-of-the-art mmWave testbed and an AV testbed in real-world settings. Various strategies are developed to take control of the victim AV by spoofing its mmWave sensing module, including adding fake obstacles at arbitrary locations and faking the locations of existing obstacles. Five real-world attack scenarios are constructed to spoof the victim AV and force it to make dangerous driving decisions leading to a fatal crash. Field experiments are conducted to study the impact of the various attack scenarios using a Lincoln MKZ-based AV testbed, which validate that the attacker can indeed assume control of the victim AV to compromise its security and safety. To defend the attacks, we design and implement a challenge-response authentication scheme and a RF fingerprinting scheme to reliably detect aforementioned spoofing attacks.
Millimeter-wave wireless networks offer high throughput and can (ideally) prevent eavesdropping attacks using narrow, directional beams. Unfortunately, imperfections in physical hardware mean today's antenna arrays all exhibit side lobes, signals that carry the same sensitive data as the main lobe. Our work presents results of the first experimental study of the security properties of mmWave transmissions against side-lobe eavesdropping attacks. We show that these attacks on mmWave links are highly effective in both indoor and outdoor settings, and they cannot be eliminated by improved hardwar…
We have seen a surge in research aims toward adversarial attacks and defenses in AI/ML systems. While it is crucial to formulate new attack methods and devise novel defense strategies for robustness, it is also imperative to recognize who is responsible for implementing, validating, and justifying the necessity of these defenses. In particular, which components of the system are vulnerable to what type of adversarial attacks, and the expertise needed to realize the severity of adversarial attacks. Also how to evaluate and address the adversarial challenges in order to recommend defense strateg…
Attack-defense trees are a novel methodology for graphical security modeling and assessment. The methodology includes visual, intuitive tree models whose analysis is supported by a rigorous mathematical formalism. Both, the intuitive and the formal components of the approach can be used for quantitative analysis of attack-defense scenarios. In practice, we use intuitive questions to ask about aspects of scenarios we are interested in. Formally, a computational procedure, defined with the help of attribute domains and a bottom-up algorithm, is applied to derive the corresponding numerical value…
UT Austin（德克萨斯大学奥斯汀分校）；MIT EECS（麻省理工学院电子工程与计算机科学系）
We propose a new type of attack for finding adversarial examples for image classifiers. Our method exploits spanners, i.e. deep neural networks whose input space is low-dimensional and whose output range approximates the set of images of interest. Spanners may be generators of GANs or decoders of VAEs. The key idea in our attack is to search over latent code pairs to find ones that generate nearby images with different classifier outputs. We argue that our attack is stronger than searching over perturbations of real images. Moreover, we show that our stronger attack can be used to reduce the a…
Michigan State University（密歇根州立大学）；The Hong Kong Polytechnic University（香港理工大学）
Federated learning is considered as an effective privacy-preserving learning mechanism that separates the client's data and model training process. However, federated learning is still under the risk of privacy leakage because of the existence of attackers who deliberately conduct gradient leakage attacks to reconstruct the client data. Recently, popular strategies such as gradient perturbation methods and input encryption methods have been proposed to defend against gradient leakage attacks. Nevertheless, these defenses can either greatly sacrifice the model performance, or be evaded by more …
Google Brain（谷歌大脑）；MIT（麻省理工学院）；University of Tübingen（图宾根大学）
Correctly evaluating defenses against adversarial examples has proven to be extremely difficult. Despite the significant amount of recent work attempting to design defenses that withstand adaptive attacks, few have succeeded; most papers that propose defenses are quickly shown to be incorrect. We believe a large contributing factor is the difficulty of performing security evaluations. In this paper, we discuss the methodological foundations, review commonly accepted best practices, and suggest new methods for evaluating defenses to adversarial examples. We hope that both researchers developing…
The Byzantine attack in cooperative spectrum sensing (CSS), also known as the spectrum sensing data falsification (SSDF) attack in the literature, is one of the key adversaries to the success of cognitive radio networks (CRNs). In the past couple of years, the research on the Byzantine attack and defense strategies has gained worldwide increasing attention. In this paper, we provide a comprehensive survey and tutorial on the recent advances in the Byzantine attack and defense for CSS in CRNs. Specifically, we first briefly present the preliminaries of CSS for general readers, including signal …
Data Poisoning (DP) is an effective attack that causes trained classifiers to misclassify their inputs. DP attacks significantly degrade a classifier's accuracy by covertly injecting attack samples into the training set. Broadly applicable to different classifier structures, without strong assumptions about the attacker, an {\it unsupervised} Bayesian Information Criterion (BIC)-based mixture model defense against "error generic" DP attacks is herein proposed that: 1) addresses the most challenging {\it embedded} DP scenario wherein, if DP is present, the poisoned samples are an {\it a priori}…
Despite numerous countermeasures proposed by practitioners and researchers, remote control-flow alteration of programs with memory-safety vulnerabilities continues to be a realistic threat. Guaranteeing that complex software is completely free of memory-safety vulnerabilities is extremely expensive. Probabilistic countermeasures that depend on random secret keys are interesting, because they are an inexpensive way to raise the bar for attackers who aim to exploit memory-safety vulnerabilities. Moreover, some countermeasures even support legacy systems. However, it is unclear how to quantify an…
University of Houston（休斯顿大学）；Washington University in St. Louis（圣路易斯华盛顿大学）
Moving target defense (MTD) is a proactive defense approach that aims to thwart attacks by continuously changing the attack surface of a system (e.g., changing host or network configurations), thereby increasing the adversary's uncertainty and attack cost. To maximize the impact of MTD, a defender must strategically choose when and what changes to make, taking into account both the characteristics of its system as well as the adversary's observed activities. Finding an optimal strategy for MTD presents a significant challenge, especially when facing a resourceful and determined adversary who m…
In spite of the enormous success of neural networks, adversarial examples remain a relatively weakly understood feature of deep learning systems. There is a considerable effort in both building more powerful adversarial attacks and designing methods to counter the effects of adversarial examples. We propose a method to transform the adversarial input data through a mixture of generators in order to recover the correct class obfuscated by the adversarial attack. A canonical set of images is used to generate adversarial examples through potentially multiple attacks. Such transformed images are p…
Arizona State University（亚利桑那州立大学）；IBM Research（IBM研究院）
Present attack methods can make state-of-the-art classification systems based on deep neural networks misclassify every adversarially modified test example. The design of general defense strategies against a wide range of such attacks still remains a challenging problem. In this paper, we draw inspiration from the fields of cybersecurity and multi-agent systems and propose to leverage the concept of Moving Target Defense (MTD) in designing a meta-defense for 'boosting' the robustness of an ensemble of deep neural networks (DNNs) for visual classification tasks against such adversarial attacks.…
Network defenses based on traditional tools, techniques, and procedures fail to account for the attacker's inherent advantage present due to the static nature of network services and configurations. To take away this asymmetric advantage, Moving Target Defense (MTD) continuously shifts the configuration of the underlying system, in turn reducing the success rate of cyberattacks. In this survey, we analyze the recent advancements made in the development of MTDs and define categorizations that capture the key aspects of such defenses. We first categorize these defenses into different sub-classes…
University of California Merced（加州大学默塞德分校）；Tagliatela College of Engineering（塔利亚泰拉工程学院）；University of New Haven（纽黑文大学）
State-of-the-art machine learning models are vulnerable to data poisoning attacks whose purpose is to undermine the integrity of the model. However, the current literature on data poisoning attacks is mainly focused on ad hoc techniques that are only applicable to specific machine learning models. Additionally, the existing data poisoning attacks in the literature are limited to either binary classifiers or to gradient-based algorithms. To address these limitations, this paper first proposes a novel model-free label-flipping attack based on the multi-modality of the data, in which the adversar…
The impact of citizen journalism raises important video integrity and credibility issues. In this article, we introduce Vamos, the first user transparent video "liveness" verification solution based on video motion, that accommodates the full range of camera movements, and supports videos of arbitrary length. Vamos uses the agreement between video motion and camera movement to corroborate the video authenticity. Vamos can be integrated into any mobile video capture application without requiring special user training. We develop novel attacks that target liveness verification solutions. The att…
Since the discovery of adversarial attacks against machine learning models nearly a decade ago, research on adversarial machine learning has rapidly evolved into an eternal war between defenders, who seek to increase the robustness of ML models against adversarial attacks, and adversaries, who seek to develop better attacks capable of weakening or defeating these defenses. This domain, however, has found little buy-in from ML practitioners, who are neither overtly concerned about these attacks affecting their systems in the real world nor are willing to trade off the accuracy of their models i…
Federated learning is a recently proposed paradigm that enables multiple clients to collaboratively train a joint model. It allows clients to train models locally, and leverages the parameter server to generate a global model by aggregating the locally submitted gradient updates at each round. Although the incentive model for federated learning has not been fully developed, it is supposed that participants are able to get rewards or the privilege to use the final global model, as a compensation for taking efforts to train the model. Therefore, a client who does not have any local data has the …
Caller ID parodying produces the valid Caller character, in this manner deciding seem to start from another client. This apparently basic assault strategy has been utilized in the developing communication fake and trick calls, bringing about significant financial trouble. Unfortunately, callerID spoofing is easy to implement but yet it is difficult to have protection against it. In addition, there are not effective and defense solutions available right now. In this research it is suggested the CIVE (Callee Inference & VErification), a compelling and viable guard against Caller ID spoofing. Thi…
City University of Hong Kong（香港城市大学）；Monash University（莫纳什大学）；Tencent（腾讯）；Nanyang Technological University（南洋理工大学）
Semi-supervised Federated Learning (SSFL) has recently drawn much attention due to its practical consideration, i.e., the clients may only have unlabeled data. In practice, these SSFL systems implement semi-supervised training by assigning a "guessed" label to the unlabeled data near the labeled data to convert the unsupervised problem into a fully supervised problem. However, the inherent properties of such semi-supervised training techniques create a new attack surface. In this paper, we discover and reveal a simple yet powerful poisoning attack against SSFL. Our attack utilizes the natural …
The alarming rise in the quantity of malware in the last few years poses a serious challenge to the security community and requires urgent response. However, current countermeasures seem to be no longer effective. Thus, it is our belief that it is now time for researchers and security experts to turn to nature in the search for novel inspirations for defense systems. Nature has provided species with a whole range of offensive and defensive techniques, which have been developing and improving in the course of billions of years of evolution. The extremely diverse living conditions have promoted …
Secure Confidential Business Information When Sharing Machine Learning Models
School of Management, Fudan University（复旦大学管理学院）；School of Management and Economics, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）经管学院）；Lerner College of Business and Economics, University of Delaware（特拉华大学勒纳商业与经济学院）
Cloud-based services have become part of our day-to-day software solutions. The identity authentication process is considered to be the main gateway to these services. As such, these gates have become increasingly susceptible to aggressive attackers, who may use Denial of Service (DoS) attacks to close these gates permanently. There are a number of authentication protocols that are strong enough to verify identities and protect traditional networked applications. However, these authentication protocols may themselves introduce DoS risks when used in cloud-based applications. This risk introduc…
Security games model the confrontation between a defender protecting a set of targets and an attacker who tries to capture them. A variant of these games assumes security interdependence between targets, facilitating contagion of an attack. So far only stochastic spread of an attack has been considered. In this work, we introduce a version of security games, where the attacker strategically drives the entire spread of attack and where interconnections between nodes affect their susceptibility to be captured. We find that the strategies effective in the settings without contagion or with stocha…
Recent results suggest that attacks against supervised machine learning systems are quite effective, while defenses are easily bypassed by new attacks. However, the specifications for machine learning systems currently lack precise adversary definitions, and the existing attacks make diverse, potentially unrealistic assumptions about the strength of the adversary who launches them. We propose the FAIL attacker model, which describes the adversary's knowledge and control along four dimensions. Our model allows us to consider a wide range of weaker adversaries who have limited control and incomp…
Adversarial Diffusion Across Modalities: A Fusion Survey of Attacks, Defenses, and Evaluation for Text, Vision, and Vision-Language Models
Information and Computer Science Department, King Fahd University of Petroleum & Minerals（国王法赫德石油矿物大学信息与计算机科学系）；College of Computer Science and Information Technology, Imam Abdulrahman Bin Faisal University（伊玛目阿卜杜勒拉赫曼·本·法伊塞尔大学计算机科学与信息科技学院）；SDAIA-KFUPM Joint Research Center for Artificial Intelligence, King Fahd University of Petroleum & Minerals（SDAIA-KFUPM人工智能联合研究中心，国王法赫德石油矿物大学）
We find that the conventional belief of vehicle cyber attacks and their defenses---attacks are feasible and thus defenses are required only when the vehicle's ignition is turned on---does not hold. We verify this fact by discovering and applying two new practical and important attacks: battery-drain and Denial-of-Body-control (DoB). The former can drain the vehicle battery while the latter can prevent the owner from starting or even opening/entering his car, when either or both attacks are mounted with the ignition off. We first analyze how operation (e.g., normal, sleep, listen) modes of ECUs…
Neural network pruning has been an essential technique to reduce the computation and memory requirements for using deep neural networks for resource-constrained devices. Most existing research focuses primarily on balancing the sparsity and accuracy of a pruned neural network by strategically removing insignificant parameters and retraining the pruned model. Such efforts on reusing training samples pose serious privacy risks due to increased memorization, which, however, has not been investigated yet. In this paper, we conduct the first analysis of privacy risks in neural network pruning. Spec…
Federated learning (FL) provides autonomy and privacy by design to participating peers, who cooperatively build a machine learning (ML) model while keeping their private data in their devices. However, that same autonomy opens the door for malicious peers to poison the model by conducting either untargeted or targeted poisoning attacks. The label-flipping (LF) attack is a targeted poisoning attack where the attackers poison their training data by flipping the labels of some examples from one class (i.e., the source class) to another (i.e., the target class). Unfortunately, this attack is easy …
