Assisted-navigation applications have a relevant impact on our daily life. However, technological progress in virtualization technologies and Software-Defined Radios recently enabled new attack vectors, namely, road traffic poisoning. These attacks open up several dreadful scenarios, which are addressed in this contribution by identifying the associated challenges and proposing innovative countermeasures.
Jashore University of Science and Technology（杰索尔科技大学）；Deakin University（迪肯大学）；RMIT University（皇家墨尔本理工大学）
Accurate building energy prediction is useful in various applications starting from building energy automation and management to optimal storage control. However, vulnerabilities should be considered when designing building energy prediction models, as intelligent attackers can deliberately influence the model performance using sophisticated attack models. These may consequently degrade the prediction accuracy, which may affect the efficiency and performance of the building energy management systems. In this paper, we investigate the impact of bi-level poisoning attacks on regression models of…
Manipulating Recommender Systems: A Survey of Poisoning Attacks and Countermeasures
Griffith University（格里菲斯大学）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；Monash University（莫纳什大学）；Humboldt-Universität zu Berlin（柏林洪堡大学）；The University of Queensland（昆士兰大学）
The recent advancements in machine learning have led to a wave of interest in adopting online learning-based approaches for long-standing attack mitigation issues. In particular, DDoS attacks remain a significant threat to network service availability even after more than two decades. These attacks have been well studied under the assumption that malicious traffic originates from a single attack profile. Based on this premise, malicious traffic characteristics are assumed to be considerably different from legitimate traffic. Consequently, online filtering methods are designed to learn network …
Northeastern University（东北大学）；University of Cagliari（卡利亚里大学）；Pluribus One（普卢里布斯一号）；UC Berkeley（加州大学伯克利分校）
As machine learning becomes widely used for automated decisions, attackers have strong incentives to manipulate the results and models generated by machine learning algorithms. In this paper, we perform the first systematic study of poisoning attacks and their countermeasures for linear regression models. In poisoning attacks, attackers deliberately influence the training data to manipulate the results of a predictive model. We propose a theoretically-grounded optimization framework specifically designed for linear regression and demonstrate its effectiveness on a range of datasets and models.…
Trilateration has recently become one of the well-known threat models to the user's location privacy in location-based applications (aka: location-based services or LBS), especially those containing highly sensitive information such as dating applications. The threat model mainly depends on the distance shown from the targeted victim to the adversary to pinpoint the victim's position. As a countermeasure, most of location-based applications have already implemented the "hide distance" function to protect their user's location privacy. The effectiveness of such approaches however is still quest…
Trilateration is one of the well-known threat models to the user's location privacy in location-based apps, especially those contain highly sensitive information such as dating apps. The threat model mainly bases on the publicly shown distance from a targeted victim to the adversary to pinpoint the victim's location. As a countermeasure, most of location-based apps have already implemented the 'hide distance' function, or added noise to the publicly shown distance in order to protect their user's location privacy. The effectiveness of such approaches however is still questionable.
Indiscriminate data poisoning attacks are quite effective against supervised learning. However, not much is known about their impact on unsupervised contrastive learning (CL). This paper is the first to consider indiscriminate poisoning attacks of contrastive learning. We propose Contrastive Poisoning (CP), the first effective such attack on CL. We empirically show that Contrastive Poisoning, not only drastically reduces the performance of CL algorithms, but also attacks supervised learning models, making it the most generalizable indiscriminate poisoning attack. We also show that CL algorithm…
Ruhr University Bochum（波鸿鲁尔大学）；imec-COSIC KU Leuven（鲁汶大学imec-COSIC研究中心）
In this paper we present LiM ("Less is More"), a malware classification framework that leverages Federated Learning to detect and classify malicious apps in a privacy-respecting manner. Information about newly installed apps is kept locally on users' devices, so that the provider cannot infer which apps were installed by users. At the same time, input from all users is taken into account in the federated learning process and they all benefit from better classification performance. A key challenge of this setting is that users do not have access to the ground truth (i.e. they cannot correctly i…
Universitat Pomepu Fabra（庞培法布拉大学）；Università degli Studi di Cagliari（卡利亚里大学）
Research in adversarial machine learning has shown how the performance of machine learning models can be seriously compromised by injecting even a small fraction of poisoning points into the training data. While the effects on model accuracy of such poisoning attacks have been widely studied, their potential effects on other model performance metrics remain to be evaluated. In this work, we introduce an optimization framework for poisoning attacks against algorithmic fairness, and develop a gradient-based poisoning attack aimed at introducing classification disparities among different groups i…
Fair classification aims to stress the classification models to achieve the equality (treatment or prediction quality) among different sensitive groups. However, fair classification can be under the risk of poisoning attacks that deliberately insert malicious training samples to manipulate the trained classifiers' performance. In this work, we study the poisoning scenario where the attacker can insert a small fraction of samples into training data, with arbitrary sensitive attributes as well as other predictive features. We demonstrate that the fairly trained classifiers can be greatly vulnera…
As in-the-wild data are increasingly involved in the training stage, machine learning applications become more susceptible to data poisoning attacks. Such attacks typically lead to test-time accuracy degradation or controlled misprediction. In this paper, we investigate the third type of exploitation of data poisoning - increasing the risks of privacy leakage of benign training samples. To this end, we demonstrate a set of data poisoning attacks to amplify the membership exposure of the targeted class. We first propose a generic dirty-label attack for supervised classification algorithms. We t…
University of Tartu（塔尔图大学）；Telefonica Research（西班牙电信研究院）
Machine Learning (ML)-powered apps are used in pervasive devices such as phones, tablets, smartwatches and IoT devices. Recent advances in collaborative, distributed ML such as Federated Learning (FL) attempt to solve privacy concerns of users and data owners, and thus used by tech industry leaders such as Google, Facebook and Apple. However, FL systems and models are still vulnerable to adversarial membership and attribute inferences and model poisoning attacks, especially in FL-as-a-Service ecosystems recently proposed, which can enable attackers to access multiple ML-powered apps. In this w…
Machine learning algorithms are used to construct a mathematical model for a system based on training data. Such a model is capable of making highly accurate predictions without being explicitly programmed to do so. These techniques have a great many applications in all areas of the modern digital economy and artificial intelligence. More importantly, these methods are essential for a rapidly increasing number of safety-critical applications such as autonomous vehicles and intelligent defense systems. However, emerging adversarial learning attacks pose a serious security threat that greatly un…
Deep learning models have achieved high performance on many tasks, and thus have been applied to many security-critical scenarios. For example, deep learning-based face recognition systems have been used to authenticate users to access many security-sensitive applications like payment apps. Such usages of deep learning systems provide the adversaries with sufficient incentives to perform attacks against these systems for their adversarial purposes. In this work, we consider a new type of attacks, called backdoor attacks, where the attacker's goal is to create a backdoor into a learning-based a…
University of Adelaide（阿德莱德大学）；CSIRO’s Data61（澳大利亚联邦科学与工业研究组织Data61）；Cyber Security Cooperative Research Centre（网络安全合作研究中心）
Context: Research at the intersection of cybersecurity, Machine Learning (ML), and Software Engineering (SE) has recently taken significant steps in proposing countermeasures for detecting sophisticated data exfiltration attacks. It is important to systematically review and synthesize the ML-based data exfiltration countermeasures for building a body of knowledge on this important topic. Objective: This paper aims at systematically reviewing ML-based data exfiltration countermeasures to identify and classify ML approaches, feature engineering techniques, evaluation datasets, and performance me…
University of Pittsburgh（匹兹堡大学）；Air Force Research Laboratory（空军研究实验室）；Duke University（杜克大学）
Poisoning attack is identified as a severe security threat to machine learning algorithms. In many applications, for example, deep neural network (DNN) models collect public data as the inputs to perform re-training, where the input data can be poisoned. Although poisoning attack against support vector machines (SVM) has been extensively studied before, there is still very limited knowledge about how such attack can be implemented on neural networks (NN), especially DNNs. In this work, we first examine the possibility of applying traditional gradient-based method (named as the direct gradient …
Hard Work Does Not Always Pay Off: Poisoning Attacks on Neural Architecture Search
School of Electrical Engineering and Computer Science（电气工程与计算机科学学院）；Oregon State University（俄勒冈州立大学）；College of Information Science and Technology（信息科学与技术学院）；Penn State University（宾夕法尼亚州立大学）
Nanjing University of Science and Technology（南京理工大学）；Data61, CSIRO（澳大利亚联邦科学与工业研究组织 Data61）；The University of Adelaide（阿德莱德大学）；The University of Queensland（昆士兰大学）；Hunan University（湖南大学）；Sungkyunkwan University（成均馆大学）
This work provides the community with a timely comprehensive review of backdoor attacks and countermeasures on deep learning. According to the attacker's capability and affected stage of the machine learning pipeline, the attack surfaces are recognized to be wide and then formalized into six categorizations: code poisoning, outsourcing, pretrained, data collection, collaborative learning and post-deployment. Accordingly, attacks under each categorization are combed. The countermeasures are categorized into four general classes: blind backdoor removal, offline backdoor inspection, online backdo…
Local Differential Privacy (LDP) protocols enable an untrusted data collector to perform privacy-preserving data analytics. In particular, each user locally perturbs its data to preserve privacy before sending it to the data collector, who aggregates the perturbed data to obtain statistics of interest. In the past several years, researchers from multiple communities -- such as security, database, and theoretical computer science -- have proposed many LDP protocols. These studies mainly focused on improving the utility of the LDP protocols. However, the security of LDP protocols is largely unex…
In this work, we investigate the concept of biometric backdoors: a template poisoning attack on biometric systems that allows adversaries to stealthily and effortlessly impersonate users in the long-term by exploiting the template update procedure. We show that such attacks can be carried out even by attackers with physical limitations (no digital access to the sensor) and zero knowledge of training data (they know neither decision boundaries nor user template). Based on the adversaries' own templates, they craft several intermediate samples that incrementally bridge the distance between their…
Android virtualization enables an app to create a virtual environment, in which other apps can run. Originally designed to overcome the limitations of mobile apps dimensions, malicious developers soon started exploiting this technique to design novel attacks. As a consequence, researchers proposed new defence mechanisms that enable apps to detect whether they are running in a virtual environment. In this paper, we propose Mascara, the first attack that exploits the virtualization technique in a new way, achieving the full feasibility against any Android app and proving the ineffectiveness of e…
Limited by the small keyboard, most mobile apps support the automatic login feature for better user experience. Therefore, users avoid the inconvenience of retyping their ID and password when an app runs in the foreground again. However, this auto-login function can be exploited to launch the so-called "data-clone attack": once the locally-stored, auto-login depended data are cloned by attackers and placed into their own smartphones, attackers can break through the login-device number limit and log in to the victim's account stealthily. A natural countermeasure is to check the consistency of d…
Data Poisoning Attacks to Locally Differentially Private Range Query Protocols
National Yang Ming Chiao Tung University（国立阳明交通大学）；RIKEN AIP（理化学研究所人工智能项目）；ISM and AIST（信息科学研究机构与产业技术综合研究所）；Institute of Science Tokyo（东京科学大学）；Meiji University（明治大学）
Nowadays, the usage of smartphones and their applications have become rapidly increasing popular in people's daily life. Over the last decade, availability of mobile money services such as mobile-payment systems and app markets have significantly increased due to the different forms of apps and connectivity provided by mobile devices such as 3G, 4G, GPRS, and Wi-Fi, etc. In the same trend, the number of vulnerabilities targeting these services and communication networks has raised as well. Therefore, smartphones have become ideal target devices for malicious programmers. With increasing the nu…
