投毒后门攻击指攻击者通过在推理阶段向信号中植入触发器，操纵训练数据以在受害模型中诱导特定行为的攻击方式。我们将不修改训练标签的干净标签后门（CLBD，Clean Label Backdoor）数据投毒攻击适配于支持/执行口语理解（Spoken Language Understanding，SLU）任务的最先进语音识别模型，通过投毒10%的训练数据实现了99.8%的攻击成功率。我们分析了投毒信号强度、投毒样本占比以及触发器选择的变化对攻击效果的影响。我们还发现，当CLBD攻击应用于对代理模型而言本身难以处理的训练样本时，攻击成功率最高。采用这一策略，我们仅投毒了少至1.5%的训练数据，就实现了99.3%的攻击成功率。最后，我们应用了两种先前开发的针对基于梯度的攻击的防御方法，发现它们对投毒攻击的防御效果好坏参半。
英文摘要：
Poisoning backdoor attacks involve an adversary manipulating the training data to induce certain behaviors in the victim model by inserting a trigger in the signal at inference time. We adapted clean label backdoor (CLBD)-data poisoning attacks, which do not modify the training labels, on state-of-the-art speech recognition models that support/perform a Spoken Language Understanding task, achieving 99.8% attack success rate by poisoning 10% of the training data. We analyzed how varying the signal-strength of the poison, percent of samples poisoned, and choice of trigger impact the attack. We also found that CLBD attacks are most successful when applied to training samples that are inherently hard for a proxy model. Using this strategy, we achieved an attack success rate of 99.3% by poisoning a meager 1.5% of the training data. Finally, we applied two previously developed defenses against gradient-based attacks, and found that they attain mixed success against poisoning.
Center for Machine Learning, University of Maryland - College Park（马里兰大学帕克分校机器学习中心）
Targeted clean-label data poisoning is a type of adversarial attack on machine learning systems in which an adversary injects a few correctly-labeled, minimally-perturbed samples into the training data, causing a model to misclassify a particular test sample during inference. Although defenses have been proposed for general poisoning attacks, no reliable defense for clean-label attacks has been demonstrated, despite the attacks' effectiveness and realistic applications. In this work, we propose a simple, yet highly-effective Deep k-NN defense against both feature collision and convex polytope …
Data poisoning attacks compromise the integrity of machine-learning models by introducing malicious training samples to influence the results during test time. In this work, we investigate backdoor data poisoning attack on deep neural networks (DNNs) by inserting a backdoor pattern in the training images. The resulting attack will misclassify poisoned test samples while maintaining high accuracies for the clean test-set. We present two approaches for detection of such poisoned samples by quantifying the uncertainty estimates associated with the trained models. In the first approach, we model t…
Huazhong University of Science and Technology（华中科技大学）；Wuhan University of Technology（武汉理工大学）；Georgia Institute of Technology（佐治亚理工学院）
Machine learning techniques have been widely applied to various applications. However, they are potentially vulnerable to data poisoning attacks, where sophisticated attackers can disrupt the learning procedure by injecting a fraction of malicious samples into the training dataset. Existing defense techniques against poisoning attacks are largely attack-specific: they are designed for one specific type of attacks but do not work for other types, mainly due to the distinct principles they follow. Yet few general defense strategies have been developed. In this paper, we propose De-Pois, an attac…
Recent exploration of the multi-agent backdoor attack demonstrated the backfiring effect, a natural defense against backdoor attacks where backdoored inputs are randomly classified. This yields a side-effect of low accuracy w.r.t. clean labels, which motivates this paper's work on the construction of multi-agent backdoor defenses that maximize accuracy w.r.t. clean labels and minimize that of poison labels. Founded upon agent dynamics and low-loss subspace construction, we contribute three defenses that yield improved multi-agent backdoor robustness.
University of Maryland, College Park（马里兰大学帕克分校）；United States Naval Academy（美国海军学院）；Cornell University（康奈尔大学）
Clean-label poisoning attacks inject innocuous looking (and "correctly" labeled) poison images into training data, causing a model to misclassify a targeted image after being trained on this data. We consider transferable poisoning attacks that succeed without access to the victim network's outputs, architecture, or (in some cases) training data. To achieve this, we propose a new "polytope attack" in which poison images are designed to surround the targeted image in feature space. We also demonstrate that using Dropout during poison creation helps to enhance transferability of this attack. We …
University of California, Santa Barbara（加州大学圣巴巴拉分校）
A recent source of concern for the security of neural networks is the emergence of clean-label dataset poisoning attacks, wherein correctly labeled poison samples are injected into the training dataset. While these poison samples look legitimate to the human observer, they contain malicious characteristics that trigger a targeted misclassification during inference. We propose a scalable and transferable clean-label poisoning attack against transfer learning, which creates poison images with their center close to the target image in the feature space. Our attack, Bullseye Polytope, improves the…
Virginia Tech（弗吉尼亚理工大学）；Sony AI（索尼人工智能）；Texas A&M University-Commerce（得克萨斯农工大学康莫斯分校）
Backdoor attacks insert malicious data into a training set so that, during inference time, it misclassifies inputs that have been patched with a backdoor trigger as the malware specified label. For backdoor attacks to bypass human inspection, it is essential that the injected data appear to be correctly labeled. The attacks with such property are often referred to as "clean-label attacks." Existing clean-label backdoor attacks require knowledge of the entire training set to be effective. Obtaining such knowledge is difficult or impossible because training data are often gathered from multiple …
Wicked Oddities: Selectively Poisoning for Effective Clean-Label Backdoor Attacks
College of Engineering and Computer Science, VinUniversity（VinUniversity工程与计算机科学学院）；CSIRO’s Data61（澳大利亚联邦科学与工业研究组织Data61）；Johns Hopkins University（约翰斯·霍普金斯大学）；Vietnam National University（越南国立大学）
Generalization Bound and New Algorithm for Clean-Label Backdoor Attack
Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Kaiyuan International Mathematical Sciences Institute（开源国际数学科学研究所）
Backdoor attacks threaten Deep Neural Networks (DNNs). Towards stealthiness, researchers propose clean-label backdoor attacks, which require the adversaries not to alter the labels of the poisoned training datasets. Clean-label settings make the attack more stealthy due to the correct image-label pairs, but some problems still exist: first, traditional methods for poisoning training data are ineffective; second, traditional triggers are not stealthy which are still perceptible. To solve these problems, we propose a two-phase and image-specific triggers generation method to enhance clean-label …
Although Deep Neural Network (DNN) has led to unprecedented progress in various natural language processing (NLP) tasks, research shows that deep models are extremely vulnerable to backdoor attacks. The existing backdoor attacks mainly inject a small number of poisoned samples into the training dataset with the labels changed to the target one. Such mislabeled samples would raise suspicion upon human inspection, potentially revealing the attack. To improve the stealthiness of textual backdoor attacks, we propose the first clean-label framework Kallima for synthesizing mimesis-style backdoor sa…
Checkerboard: Closed-Form and Data-Independent Trigger Design for Clean-Label Backdoor Attacks
School of Computer Science and Information Engineering（计算机科学与信息工程学院）；Hefei University of Technology（合肥工业大学）；School of Information Science and Technology（信息科学与技术学院）；University of Science and Technology of China（中国科学技术大学）；Faculty of Business Data Science（商务数据科学学院）；Kansai University（大阪大学）；RIKEN Center for Advanced Intelligence Project（日本RIKEN先进人工智能研究中心）；The University of Electro-Communications（电通大学）；Agency for Science, Technology and Research (A*STAR)（科技研究局（A*STAR））
We propose CLEANN, the first end-to-end framework that enables online mitigation of Trojans for embedded Deep Neural Network (DNN) applications. A Trojan attack works by injecting a backdoor in the DNN while training; during inference, the Trojan can be activated by the specific backdoor trigger. What differentiates CLEANN from the prior work is its lightweight methodology which recovers the ground-truth class of Trojan samples without the need for labeled data, model retraining, or prior assumptions on the trigger or the attack. We leverage dictionary learning and sparse approximation to char…
Ouroboros: Self-Referential Backdoor Attacks on Speech Enhancement via Clean Audio Triggers
Faculty of Electrical Engineering and Computer Science, Ningbo University（宁波大学电气工程与计算机科学学院）；College of Artificial Intelligence, Ningbo University of Finance and Economics（宁波财经学院人工智能学院）
We propose a stealthy clean-label video backdoor attack against Deep Learning (DL)-based models aiming at detecting a particular class of spoofing attacks, namely video rebroadcast attacks. The injected backdoor does not affect spoofing detection in normal conditions, but induces a misclassification in the presence of a specific triggering signal. The proposed backdoor relies on a temporal trigger altering the average chrominance of the video sequence. The backdoor signal is designed by taking into account the peculiarities of the Human Visual System (HVS) to reduce the visibility of the trigg…
University of Maryland（马里兰大学）；Cornell University（康奈尔大学）
Data poisoning is an attack on machine learning models wherein the attacker adds examples to the training set to manipulate the behavior of the model at test time. This paper explores poisoning attacks on neural nets. The proposed attacks use "clean-labels"; they don't require the attacker to have any control over the labeling of training data. They are also targeted; they control the behavior of the classifier on a $\textit{specific}$ test instance without degrading overall classifier performance. For example, an attacker could add a seemingly innocuous image (that is properly labeled) to a t…
Backdoor attack intends to embed hidden backdoor into deep neural networks (DNNs), such that the attacked model performs well on benign samples, whereas its prediction will be maliciously changed if the hidden backdoor is activated by the attacker defined trigger. This threat could happen when the training process is not fully controlled, such as training on third-party data-sets or adopting third-party models. There has been a lot of research and different methods to defend such type of backdoor attacks, one being robustness-aware perturbation-based defense method. This method mainly exploits…
University of Michigan（密歇根大学）；NVIDIA（英伟达）；California Institute of Technology（加州理工学院）；ASU（亚利桑那州立大学）；Stanford University（斯坦福大学）
Trajectory prediction using deep neural networks (DNNs) is an essential component of autonomous driving (AD) systems. However, these methods are vulnerable to adversarial attacks, leading to serious consequences such as collisions. In this work, we identify two key ingredients to defend trajectory prediction models against adversarial attacks including (1) designing effective adversarial training methods and (2) adding domain-specific data augmentation to mitigate the performance degradation on clean data. We demonstrate that our method is able to improve the performance by 46% on adversarial …
Zhejiang University（浙江大学）；Nanyang Technological University（南洋理工大学）；Shanghai Institute for Advanced Study of Zhejiang University（浙江大学上海高等研究院）；Shanghai AI Laboratory（上海人工智能实验室）；Chongqing University（重庆大学）；Peng Cheng Laboratory（鹏城实验室）；Peking University（北京大学）
Backdoor attacks pose a new threat to NLP models. A standard strategy to construct poisoned data in backdoor attacks is to insert triggers (e.g., rare words) into selected sentences and alter the original label to a target label. This strategy comes with a severe flaw of being easily detected from both the trigger and the label perspectives: the trigger injected, which is usually a rare word, leads to an abnormal natural language expression, and thus can be easily detected by a defense model; the changed target label leads the example to be mistakenly labeled and thus can be easily detected by…
Deep Learning backdoor attacks have a threat model similar to traditional cyber attacks. Attack forensics, a critical counter-measure for traditional cyber attacks, is hence of importance for defending model backdoor attacks. In this paper, we propose a novel model backdoor forensics technique. Given a few attack samples such as inputs with backdoor triggers, which may represent different types of backdoors, our technique automatically decomposes them to clean inputs and the corresponding triggers. It then clusters the triggers based on their properties to allow automatic attack categorization…
Wuhan University（武汉大学）；RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）；Xi’an Jiaotong University（西安交通大学）
People are not always receptive to their voice data being collected and misused. Training the audio intelligence systems needs these data to build useful features, but the cost for getting permissions or purchasing data is very high, which inevitably encourages hackers to collect these voice data without people's awareness. To discourage the hackers from proactively collecting people's voice data, we are the first to propose a clean-label poisoning attack, called WaveFuzz, which can prevent intelligence audio models from building useful features from protected (poisoned) voice data but still p…
Temporal Poisoning: Clean-Label Backdoors via Event Redistribution in SNNs
Radboud University（拉德堡德大学）；Ikerlan Research Centre（伊克尔兰研究中心）；University of Bergen（卑尔根大学）；University of Zagreb（萨格勒布大学）；Faculty of Electrical Engineering and Computing（电气工程与计算机学院）
