Challenges and Countermeasures for Adversarial Attacks on Deep Reinforcement Learning
Information Technology University (ITU)（信息技术大学）；Qatar University（卡塔尔大学）；Hamad bin Khalifa University (HBKU)（哈马德·本·哈利法大学）；University of Technology Sydney (UTS)（悉尼科技大学）；Nanyang Technological University (NTU)（南洋理工大学）
Inaam Ilahi, Muhammad Usama, Junaid Qadir, Muhammad Umar Janjua, Ala Al-Fuqaha, Dinh Thai Hoang, Dusit Niyato
Deep Reinforcement Learning (DRL) has numerous applications in the real world thanks to its outstanding ability in quickly adapting to the surrounding environments. Despite its great advantages, DRL is susceptible to adversarial attacks, which precludes its use in real-life critical systems and applications (e.g., smart grids, traffic controls, and autonomous vehicles) unless its vulnerabilities are addressed and mitigated. Thus, this paper provides a comprehensive survey that discusses emerging attacks in DRL-based systems and the potential countermeasures to defend against these attacks. We first cover some fundamental backgrounds about DRL and present emerging adversarial attacks on machine learning techniques. We then investigate more details of the vulnerabilities that the adversary can exploit to attack DRL along with the state-of-the-art countermeasures to prevent such attacks. Finally, we highlight open issues and research challenges for developing solutions to deal with attacks for DRL-based intelligent systems.
National Taiwan University（台湾大学）；The Chinese University of Hong Kong（香港中文大学）
Various forefront countermeasure methods for automatic speaker verification (ASV) with considerable performance in anti-spoofing are proposed in the ASVspoof 2019 challenge. However, previous work has shown that countermeasure models are vulnerable to adversarial examples indistinguishable from natural data. A good countermeasure model should not only be robust against spoofing audio, including synthetic, converted, and replayed audios; but counteract deliberately generated examples by malicious adversaries. In this work, we introduce a passive defense method, spatial smoothing, and a proactiv…
Jashore University of Science and Technology（杰索尔科技大学）；Deakin University（迪肯大学）；RMIT University（皇家墨尔本理工大学）
Accurate building energy prediction is useful in various applications starting from building energy automation and management to optimal storage control. However, vulnerabilities should be considered when designing building energy prediction models, as intelligent attackers can deliberately influence the model performance using sophisticated attack models. These may consequently degrade the prediction accuracy, which may affect the efficiency and performance of the building energy management systems. In this paper, we investigate the impact of bi-level poisoning attacks on regression models of…
Nanjing University of Science and Technology（南京理工大学）；Data61, CSIRO（澳大利亚联邦科学与工业研究组织 Data61）；The University of Adelaide（阿德莱德大学）；The University of Queensland（昆士兰大学）；Hunan University（湖南大学）；Sungkyunkwan University（成均馆大学）
This work provides the community with a timely comprehensive review of backdoor attacks and countermeasures on deep learning. According to the attacker's capability and affected stage of the machine learning pipeline, the attack surfaces are recognized to be wide and then formalized into six categorizations: code poisoning, outsourcing, pretrained, data collection, collaborative learning and post-deployment. Accordingly, attacks under each categorization are combed. The countermeasures are categorized into four general classes: blind backdoor removal, offline backdoor inspection, online backdo…
A Survey of Privacy-Preserving Model Explanations: Privacy Risks, Attacks, and Countermeasures
Griffith University（格里菲斯大学）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；University of Bremen（不来梅大学）；Hanoi University of Science and Technology（河内科技大学）；The University of Queensland（昆士兰大学）
The forecast of electrical loads is essential for the planning and operation of the power system. Recently, advances in deep learning have enabled more accurate forecasts. However, deep neural networks are prone to adversarial attacks. Although most of the literature focuses on integrity-based attacks, this paper proposes availability-based adversarial attacks, which can be more easily implemented by attackers. For each forecast instance, the availability attack position is optimally solved by mixed-integer reformulation of the artificial neural network. To tackle this attack, an adversarial t…
Manipulating Recommender Systems: A Survey of Poisoning Attacks and Countermeasures
Griffith University（格里菲斯大学）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；Monash University（莫纳什大学）；Humboldt-Universität zu Berlin（柏林洪堡大学）；The University of Queensland（昆士兰大学）
Northeastern University（东北大学）；University of Cagliari（卡利亚里大学）；Pluribus One（普卢里布斯一号）；UC Berkeley（加州大学伯克利分校）
As machine learning becomes widely used for automated decisions, attackers have strong incentives to manipulate the results and models generated by machine learning algorithms. In this paper, we perform the first systematic study of poisoning attacks and their countermeasures for linear regression models. In poisoning attacks, attackers deliberately influence the training data to manipulate the results of a predictive model. We propose a theoretically-grounded optimization framework specifically designed for linear regression and demonstrate its effectiveness on a range of datasets and models.…
Management Center Innsbruck（因斯布鲁克管理中心）；Department of Computers and Engineering, Czech Technical University in Prague（布拉格捷克理工大学计算机与工程系）
Adversarial machine learning, i.e., increasing the robustness of machine learning algorithms against so-called adversarial examples, is now an established field. Yet, newly proposed methods are evaluated and compared under unrealistic scenarios where costs for adversary and defender are not considered and either all samples or no samples are adversarially perturbed. We scrutinize these assumptions and propose the advanced adversarial classification game, which incorporates all relevant parameters of an adversary and a defender. Especially, we take into account economic factors on both sides an…
The rise of machine learning as a service and model sharing platforms has raised the need of traitor-tracing the models and proof of authorship. Watermarking technique is the main component of existing methods for protecting copyright of models. In this paper, we show that distillation, a widely used transformation technique, is a quite effective attack to remove watermark embedded by existing algorithms. The fragility is due to the fact that distillation does not retain the watermark embedded in the model that is redundant and independent to the main learning task. We design ingrain in respon…
As a car becomes more connected, a countermeasure against automobile theft has become a significant task in the real world. To respond to automobile theft, data mining, biometrics, and additional authentication methods are proposed. Among current countermeasures, data mining method is one of the efficient ways to capture the owner driver's unique characteristics. To identify the owner driver from thieves, previous works applied various algorithms toward driving data. Such data mining methods utilized supervised learning, thus required labeled data set. However, it is unrealistic to gather and …
This paper presents the Speech Technology Center (STC) replay attack detection systems proposed for Automatic Speaker Verification Spoofing and Countermeasures Challenge 2017. In this study we focused on comparison of different spoofing detection approaches. These were GMM based methods, high level features extraction with simple classifier and deep learning frameworks. Experiments performed on the development and evaluation parts of the challenge dataset demonstrated stable efficiency of deep learning approaches in case of changing acoustic conditions. At the same time SVM classifier with hig…
Uppsala University（乌普萨拉大学）；Hamad bin Khalifa University（哈马德·本·哈利法大学）
We present a new machine learning-based attack that exploits network patterns to detect the presence of smart IoT devices and running services in the WiFi radio spectrum. We perform an extensive measurement campaign of data collection, and we build up a model describing the traffic patterns characterizing three popular IoT smart home devices, i.e., Google Nest Mini, Amazon Echo, and Amazon Echo Dot. We prove that it is possible to detect and identify with overwhelming probability their presence and the services running by the aforementioned devices in a crowded WiFi scenario. This work proves …
University of Adelaide（阿德莱德大学）；CSIRO’s Data61（澳大利亚联邦科学与工业研究组织Data61）；Cyber Security Cooperative Research Centre（网络安全合作研究中心）
Context: Research at the intersection of cybersecurity, Machine Learning (ML), and Software Engineering (SE) has recently taken significant steps in proposing countermeasures for detecting sophisticated data exfiltration attacks. It is important to systematically review and synthesize the ML-based data exfiltration countermeasures for building a body of knowledge on this important topic. Objective: This paper aims at systematically reviewing ML-based data exfiltration countermeasures to identify and classify ML approaches, feature engineering techniques, evaluation datasets, and performance me…
Rounding confidence score is considered trivial but a simple and effective countermeasure to stop gradient descent based image reconstruction attacks. However, its capability in the face of more sophisticated reconstruction attacks is an uninvestigated research area. In this paper, we prove that, the face reconstruction attacks based on composite faces can reveal the inefficiency of rounding policy as countermeasure. We assume that, the attacker takes advantage of face composite parts which helps the attacker to get access to the most important features of the face or decompose it to the indep…
Perception plays a pivotal role in autonomous driving systems, which utilizes onboard sensors like cameras and LiDARs (Light Detection and Ranging) to assess surroundings. Recent studies have demonstrated that LiDAR-based perception is vulnerable to spoofing attacks, in which adversaries spoof a fake vehicle in front of a victim self-driving car by strategically transmitting laser signals to the victim's LiDAR sensor. However, existing attacks suffer from effectiveness and generality limitations. In this work, we perform the first study to explore the general vulnerability of current LiDAR-bas…
Voice authentication has become an integral part in security-critical operations, such as bank transactions and call center conversations. The vulnerability of automatic speaker verification systems (ASVs) to spoofing attacks instigated the development of countermeasures (CMs), whose task is to tell apart bonafide and spoofed speech. Together, ASVs and CMs form today's voice authentication platforms, advertised as an impregnable access control mechanism. We develop the first practical attack on CMs, and show how a malicious actor may efficiently craft audio samples to bypass voice authenticati…
Deep neural networks (DNN) have achieved unprecedented success in numerous machine learning tasks in various domains. However, the existence of adversarial examples has raised concerns about applying deep learning to safety-critical applications. As a result, we have witnessed increasing interests in studying attack and defense mechanisms for DNN models on different data types, such as images, graphs and text. Thus, it is necessary to provide a systematic and comprehensive overview of the main threats of attacks and the success of corresponding countermeasures. In this survey, we review the st…
This paper contributes to the study of PUFs vulnerability against modeling attacks by evaluating the security of XOR BR PUFs, XOR TBR PUFs, and obfuscated architectures of XOR BR PUF using a simplified mathematical model and deep learning (DL) techniques. Obtained results show that DL modeling attacks could easily break the security of 4-input XOR BR PUFs and 4-input XOR TBR PUFs with modeling accuracy $\sim$ 99%. Similar attacks were executed using single-layer neural networks (NN) and support vector machines (SVM) with polynomial kernel and the obtained results showed that single NNs failed …
The Hong Kong University of Science and Technology（香港科技大学）；University of Toronto（多伦多大学）
Federated learning (FL) has increasingly been deployed, in its vertical form, among organizations to facilitate secure collaborative training over siloed data. In vertical FL (VFL), participants hold disjoint features of the same set of sample instances. Among them, only one has labels. This participant, known as the active party, initiates the training and interacts with the other participants, known as the passive parties. Despite the increasing adoption of VFL, it remains largely unknown if and how the active party can extract feature data from the passive party, especially when training de…
