Deep neural networks (DNNs) are vulnerable to backdoor attacks which can hide backdoor triggers in DNNs by poisoning training data. A backdoored model behaves normally on clean test images, yet consistently predicts a particular target class for any test examples that contain the trigger pattern. As such, backdoor attacks are hard to detect, and have raised severe security concerns in real-world applications. Thus far, backdoor research has mostly been conducted in the image domain with image classification models. In this paper, we show that existing image backdoor attacks are far less effective on videos, and outline 4 strict conditions where existing attacks are likely to fail: 1) scenarios with more input dimensions (eg. videos), 2) scenarios with high resolution, 3) scenarios with a large number of classes and few examples per class (a "sparse dataset"), and 4) attacks with access to correct labels (eg. clean-label attacks). We propose the use of a universal adversarial trigger as the backdoor trigger to attack video recognition models, a situation where backdoor attacks are likely to be challenged by the above 4 strict conditions. We show on benchmark video datasets that our proposed backdoor attack can manipulate state-of-the-art video models with high success rates by poisoning only a small proportion of training data (without changing the labels). We also show that our proposed backdoor attack is resistant to state-of-the-art backdoor defense/detection methods, and can even be applied to improve image backdoor attacks. Our proposed video backdoor attack not only serves as a strong baseline for improving the robustness of video models, but also provides a new perspective for more understanding more powerful backdoor attacks.
Unveiling and Mitigating Backdoor Vulnerabilities based on Unlearning Weight Changes and Backdoor Activeness
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Research Institute of Big Data（深圳大数据研究院）
Peking University（北京大学）；Tsinghua University（清华大学）；University of Tokyo（东京大学）
As deep neural networks (DNNs) are growing larger, their requirements for computational resources become huge, which makes outsourcing training more popular. Training in a third-party platform, however, may introduce potential risks that a malicious trainer will return backdoored DNNs, which behave normally on clean samples but output targeted misclassifications whenever a trigger appears at the test time. Without any knowledge of the trigger, it is difficult to distinguish or recover benign DNNs from backdoored ones. In this paper, we first identify an unexpected sensitivity of backdoored DNN…
Huazhong University of Science and Technology（华中科技大学）；Deakin University（迪肯大学）；Harbin Institute of Technology（哈尔滨工业大学）
Due to its powerful feature learning capability and high efficiency, deep hashing has achieved great success in large-scale image retrieval. Meanwhile, extensive works have demonstrated that deep neural networks (DNNs) are susceptible to adversarial examples, and exploring adversarial attack against deep hashing has attracted many research efforts. Nevertheless, backdoor attack, another famous threat to DNNs, has not been studied for deep hashing yet. Although various backdoor attacks have been proposed in the field of image classification, existing approaches failed to realize a truly imperce…
This paper presents Poisoning MorphNet, the first backdoor attack method on point clouds. Conventional adversarial attack takes place in the inference stage, often fooling a model by perturbing samples. In contrast, backdoor attack aims to implant triggers into a model during the training stage, such that the victim model acts normally on the clean data unless a trigger is present in a sample. This work follows a typical setting of clean-label backdoor attack, where a few poisoned samples (with their content tampered yet labels unchanged) are injected into the training set. The unique contribu…
Backdoor attacks threaten Deep Neural Networks (DNNs). Towards stealthiness, researchers propose clean-label backdoor attacks, which require the adversaries not to alter the labels of the poisoned training datasets. Clean-label settings make the attack more stealthy due to the correct image-label pairs, but some problems still exist: first, traditional methods for poisoning training data are ineffective; second, traditional triggers are not stealthy which are still perceptible. To solve these problems, we propose a two-phase and image-specific triggers generation method to enhance clean-label …
MakeupAttack: Feature Space Black-box Backdoor Attack on Face Recognition via Makeup Transfer
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
Xi’an Jiaotong University（西安交通大学）；Alibaba Group（阿里巴巴集团）；Wormpex AI Research（Wormpex AI研究院）
Deep neural networks (DNNs) are known to be vulnerable to both backdoor attacks as well as adversarial attacks. In the literature, these two types of attacks are commonly treated as distinct problems and solved separately, since they belong to training-time and inference-time attacks respectively. However, in this paper we find an intriguing connection between them: for a model planted with backdoors, we observe that its adversarial examples have similar behaviors as its triggered images, i.e., both activate the same subset of DNN neurons. It indicates that planting a backdoor into a model wil…
Virginia Tech（弗吉尼亚理工大学）；Princeton University（普林斯顿大学）；University of Michigan（密歇根大学）；Samsung Research America（三星美国研究院）；Sony AI（索尼人工智能研究所）
Many backdoor removal techniques in machine learning models require clean in-distribution data, which may not always be available due to proprietary datasets. Model inversion techniques, often considered privacy threats, can reconstruct realistic training samples, potentially eliminating the need for in-distribution data. Prior attempts to combine backdoor removal and model inversion yielded limited results. Our work is the first to provide a thorough understanding of leveraging model inversion for effective backdoor removal by addressing key questions about reconstructed samples' properties, …
We propose a stealthy clean-label video backdoor attack against Deep Learning (DL)-based models aiming at detecting a particular class of spoofing attacks, namely video rebroadcast attacks. The injected backdoor does not affect spoofing detection in normal conditions, but induces a misclassification in the presence of a specific triggering signal. The proposed backdoor relies on a temporal trigger altering the average chrominance of the video sequence. The backdoor signal is designed by taking into account the peculiarities of the Human Visual System (HVS) to reduce the visibility of the trigg…
School of Cyber Science and Technology Sun Yat-Sen University（中山大学网络空间安全学院）；Department of Computer Science National University of Singapore（新加坡国立大学计算机科学系）
TransCAB: Transferable Clean-Annotation Backdoor to Object Detection with Natural Trigger in Real-World
The University of Adelaide（阿德莱德大学）；Data61, CSIRO（澳大利亚联邦科学与工业研究组织Data61）；Nanjing University of Science and Technology（南京理工大学）；The University of Western Australia（西澳大学）
Backdoor Cleaning without External Guidance in MLLM Fine-tuning
School of Computer Science, Wuhan University（武汉大学计算机学院）；Munich Research Center（慕尼黑研究中心）；Huawei Technologies（华为技术有限公司）；Nanyang Technological University（南洋理工大学）
A backdoor or Trojan attack is an important type of data poisoning attack against deep neural network (DNN) classifiers, wherein the training dataset is poisoned with a small number of samples that each possess the backdoor pattern (usually a pattern that is either imperceptible or innocuous) and which are mislabeled to the attacker's target class. When trained on a backdoor-poisoned dataset, a DNN behaves normally on most benign test samples but makes incorrect predictions to the target class when the test sample has the backdoor pattern incorporated (i.e., contains a backdoor trigger). Here …
Checkerboard: Closed-Form and Data-Independent Trigger Design for Clean-Label Backdoor Attacks
School of Computer Science and Information Engineering（计算机科学与信息工程学院）；Hefei University of Technology（合肥工业大学）；School of Information Science and Technology（信息科学与技术学院）；University of Science and Technology of China（中国科学技术大学）；Faculty of Business Data Science（商务数据科学学院）；Kansai University（大阪大学）；RIKEN Center for Advanced Intelligence Project（日本RIKEN先进人工智能研究中心）；The University of Electro-Communications（电通大学）；Agency for Science, Technology and Research (A*STAR)（科技研究局（A*STAR））
Anomalee Inc.（Anomalee公司）；School of EECS, Pennsylvania State University（宾夕法尼亚州立大学EECS学院）
Backdoor attacks (BAs) are an emerging form of adversarial attack typically against deep neural network image classifiers. The attacker aims to have the classifier learn to classify to a target class when test images from one or more source classes contain a backdoor pattern, while maintaining high accuracy on all clean test images. Reverse-Engineering-based Defenses (REDs) against BAs do not require access to the training set but only to an independent clean dataset. Unfortunately, most existing REDs rely on an unrealistic assumption that all classes except the target class are source classes…
CSIRO’s Data61（澳大利亚联邦科学与工业研究组织Data61）；Cyber Security Cooperative Research Centre（网络安全合作研究中心）
Biometric data, such as face images, are often associated with sensitive information (e.g medical, financial, personal government records). Hence, a data breach in a system storing such information can have devastating consequences. Deep learning is widely utilized for face recognition (FR); however, such models are vulnerable to backdoor attacks executed by malicious parties. Backdoor attacks cause a model to misclassify a particular class as a target class during recognition. This vulnerability can allow adversaries to gain access to highly sensitive data protected by biometric authenticatio…
Megatron: Evasive Clean-Label Backdoor Attacks against Vision Transformer
School of Computer Science, Wuhan University（武汉大学计算机学院）；Department of Computer Science and Engineering of Hong Kong University of Science and Technology（香港科技大学计算机科学与工程系）；College of Electrical Engineering, Zhejiang University（浙江大学电气工程学院）；School of Cyber Science and Engineering, Wuhan University（武汉大学网络安全学院）
CleanerCLIP: Fine-grained Counterfactual Semantic Augmentation for Backdoor Defense in Contrastive Learning
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；National University of Singapore（新加坡国立大学）；Nanyang Technological University（南洋理工大学）；Sun Yat-sen University-Shenzhen（中山大学（深圳））
King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学）；Saudi Data and Artificial Intelligence Authority (SDAIA)（沙特数据与人工智能管理局）
Deep neural networks (DNNs) are vulnerable to a class of attacks called "backdoor attacks", which create an association between a backdoor trigger and a target label the attacker is interested in exploiting. A backdoored DNN performs well on clean test images, yet persistently predicts an attacker-defined label for any sample in the presence of the backdoor trigger. Although backdoor attacks have been extensively studied in the image domain, there are very few works that explore such attacks in the video domain, and they tend to conclude that image backdoor attacks are less effective in the vi…
Idiap Research Institute（伊迪亚普研究所）；École Polytéchnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院（EPFL））
Backdoor attacks allow an attacker to embed functionality jeopardizing proper behavior of any algorithm, machine learning or not. This hidden functionality can remain inactive for normal use of the algorithm until activated by the attacker. Given how stealthy backdoor attacks are, consequences of these backdoors could be disastrous if such networks were to be deployed for applications as critical as border or access control. In this paper, we propose a novel backdoored network detection method based on the principle of anomaly detection, involving access to the clean part of the training data …
Deep learning models have recently shown to be vulnerable to backdoor poisoning, an insidious attack where the victim model predicts clean images correctly but classifies the same images as the target class when a trigger poison pattern is added. This poison pattern can be embedded in the training dataset by the adversary. Existing defenses are effective under certain conditions such as a small size of the poison pattern, knowledge about the ratio of poisoned training samples or when a validated clean dataset is available. Since a defender may not have such prior knowledge or resources, we pro…
Wicked Oddities: Selectively Poisoning for Effective Clean-Label Backdoor Attacks
College of Engineering and Computer Science, VinUniversity（VinUniversity工程与计算机科学学院）；CSIRO’s Data61（澳大利亚联邦科学与工业研究组织Data61）；Johns Hopkins University（约翰斯·霍普金斯大学）；Vietnam National University（越南国立大学）
Backdoor Attacks on Prompt-Driven Video Segmentation Foundation Models
Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Fujian Normal University（福建师范大学）；Jinan University（暨南大学）
Virginia Tech（弗吉尼亚理工大学）；University of Michigan（密歇根大学）
We propose a minimax formulation for removing backdoors from a given poisoned model based on a small set of clean data. This formulation encompasses much of prior work on backdoor removal. We propose the Implicit Bacdoor Adversarial Unlearning (I-BAU) algorithm to solve the minimax. Unlike previous work, which breaks down the minimax into separate inner and outer problems, our algorithm utilizes the implicit hypergradient to account for the interdependence between inner and outer optimization. We theoretically analyze its convergence and the generalizability of the robustness gained by solving…
Deep neural network (DNN) classifiers are vulnerable to backdoor attacks. An adversary poisons some of the training data in such attacks by installing a trigger. The goal is to make the trained DNN output the attacker's desired class whenever the trigger is activated while performing as usual for clean data. Various approaches have recently been proposed to detect malicious backdoored DNNs. However, a robust, end-to-end training approach, like adversarial training, is yet to be discovered for backdoor poisoned data. In this paper, we take the first step toward such methods by developing a robu…
Nanyang Technological University（南洋理工大学）；City University of Hong Kong（香港城市大学）
Deep neural networks (DNN) have shown great success in many computer vision applications. However, they are also known to be susceptible to backdoor attacks. When conducting backdoor attacks, most of the existing approaches assume that the targeted DNN is always available, and an attacker can always inject a specific pattern to the training data to further fine-tune the DNN model. However, in practice, such attack may not be feasible as the DNN model is encrypted and only available to the secure enclave. In this paper, we propose a novel black-box backdoor attack technique on face recognition …
In recent years, machine learning models have been shown to be vulnerable to backdoor attacks. Under such attacks, an adversary embeds a stealthy backdoor into the trained model such that the compromised models will behave normally on clean inputs but will misclassify according to the adversary's control on maliciously constructed input with a trigger. While these existing attacks are very effective, the adversary's capability is limited: given an input, these attacks can only cause the model to misclassify toward a single pre-defined or target class. In contrast, this paper exploits a novel b…
Model X-ray:Detecting Backdoored Models via Decision Boundary
University of Science and Technology of China（中国科学技术大学）；CFAR（计算与算法研究中心）；IHPC（高性能计算研究院）；A*STAR（新加坡科技研究局）；National University of Singapore（新加坡国立大学）；Nanyang Technological University（南洋理工大学）
本文提出 Model X-ray，一种仅利用干净输入硬标签构建二维决策边界来检测后门模型并识别目标攻击标签的轻量级方法，无需攻击信息或模型参数先验，实验证明其在不同攻击、数据集和架构上均有效且高效。
