IPBA: Imperceptible Perturbation Backdoor Attack in Federated Self-Supervised Learning
School of Information Engineering, Yangzhou University, China（扬州大学信息工程学院）；Institute of Information Engineering, Chinese Academy of Sciences, China（中国科学院信息工程研究所）；School of Computing and Artificial Intelligence, Chaohu University, China（合肥大学计算机与人工智能学院）；Tacoma School of Engineering and Technology, University of Washington, USA（华盛顿大学塔科马工程与技术学院）
Federated self-supervised learning (FSSL) combines the advantages of decentralized modeling and unlabeled representation learning, serving as a cutting-edge paradigm with strong potential for scalability and privacy preservation. Although FSSL has garnered increasing attention, research indicates that it remains vulnerable to backdoor attacks. Existing methods generally rely on visually obvious triggers, which makes it difficult to meet the requirements for stealth and practicality in real-world deployment. In this paper, we propose an imperceptible and effective backdoor attack method against FSSL, called IPBA. Our empirical study reveals that existing imperceptible triggers face a series of challenges in FSSL, particularly limited transferability, feature entanglement with augmented samples, and out-of-distribution properties. These issues collectively undermine the effectiveness and stealthiness of traditional backdoor attacks in FSSL. To overcome these challenges, IPBA decouples the feature distributions of backdoor and augmented samples, and introduces Sliced-Wasserstein distance to mitigate the out-of-distribution properties of backdoor samples, thereby optimizing the trigger generation process. Our experimental results on several FSSL scenarios and datasets show that IPBA significantly outperforms existing backdoor attack methods in performance and exhibits strong robustness under various defense mechanisms.
Rutgers University（罗格斯大学）；The University of Tennessee（田纳西大学）；Temple University（天普大学）
Recently backdoor attack has become an emerging threat to the security of deep neural network (DNN) models. To date, most of the existing studies focus on backdoor attack against the uncompressed model; while the vulnerability of compressed DNNs, which are widely used in the practical applications, is little exploited yet. In this paper, we propose to study and develop Robust and Imperceptible Backdoor Attack against Compact DNN models (RIBAC). By performing systematic analysis and exploration on the important design knobs, we propose a framework that can learn the proper trigger patterns, mod…
VinAI Research（VinAI研究院）；Hanoi University of Science and Technology（河内科技大学）；VinUniversity
With the thriving of deep learning and the widespread practice of using pre-trained networks, backdoor attacks have become an increasing security threat drawing many research interests in recent years. A third-party model can be poisoned in training to work well in normal conditions but behave maliciously when a trigger pattern appears. However, the existing backdoor attacks are all built on noise perturbation triggers, making them noticeable to humans. In this paper, we instead propose using warping-based triggers. The proposed backdoor outperforms the previous methods in a human inspection t…
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Peng Cheng Laboratory（鹏城实验室）；School of Data Science, the Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen)（香港中文大学（深圳）数据科学学院）；Shenzhen Research Institute of Big Data (SBRID)（深圳大数据研究院）
With the thriving of deep learning in processing point cloud data, recent works show that backdoor attacks pose a severe security threat to 3D vision applications. The attacker injects the backdoor into the 3D model by poisoning a few training samples with trigger, such that the backdoored model performs well on clean samples but behaves maliciously when the trigger pattern appears. Existing attacks often insert some additional points into the point cloud as the trigger, or utilize a linear transformation (e.g., rotation) to construct the poisoned point cloud. However, the effects of these poi…
Backdoor attacks are rapidly emerging threats to deep neural networks (DNNs). In the backdoor attack scenario, attackers usually implant the backdoor into the target model by manipulating the training dataset or training process. Then, the compromised model behaves normally for benign input yet makes mistakes when the pre-defined trigger appears. In this paper, we analyze the drawbacks of existing attack approaches and propose a novel imperceptible backdoor attack. We treat the trigger pattern as a special kind of noise following a multinomial distribution. A U-net-based network is employed to…
In this brief, we show that sequentially learning new information presented to a continual (incremental) learning model introduces new security risks: an intelligent adversary can introduce small amount of misinformation to the model during training to cause deliberate forgetting of a specific task or class at test time, thus creating "false memory" about that task. We demonstrate such an adversary's ability to assume control of the model by injecting "backdoor" attack samples to commonly used generative replay and regularization based continual learning approaches using continual learning ben…
A backdoor or Trojan attack is an important type of data poisoning attack against deep neural network (DNN) classifiers, wherein the training dataset is poisoned with a small number of samples that each possess the backdoor pattern (usually a pattern that is either imperceptible or innocuous) and which are mislabeled to the attacker's target class. When trained on a backdoor-poisoned dataset, a DNN behaves normally on most benign test samples but makes incorrect predictions to the target class when the test sample has the backdoor pattern incorporated (i.e., contains a backdoor trigger). Here …
Deep Neural Networks (DNNs) are ubiquitous and span a variety of applications ranging from image classification to real-time object detection. As DNN models become more sophisticated, the computational cost of training these models becomes a burden. For this reason, outsourcing the training process has been the go-to option for many DNN users. Unfortunately, this comes at the cost of vulnerability to backdoor attacks. These attacks aim to establish hidden backdoors in the DNN so that it performs well on clean samples, but outputs a particular target label when a trigger is applied to the input…
School of Electrical Engineering and Computer Science, The Pennsylvania State University（宾夕法尼亚州立大学电气工程与计算机科学学院）；Anomalee Inc.（Anomalee 公司）
Recently, a special type of data poisoning (DP) attack targeting Deep Neural Network (DNN) classifiers, known as a backdoor, was proposed. These attacks do not seek to degrade classification accuracy, but rather to have the classifier learn to classify to a target class whenever the backdoor pattern is present in a test example. Launching backdoor attacks does not require knowledge of the classifier or its training process - it only needs the ability to poison the training set with (a sufficient number of) exemplars containing a sufficiently strong backdoor pattern (labeled with the target cla…
INK: Inheritable Natural Backdoor Attack Against Model Distillation
Institute of Computer Application, China Academy of Engineering Physics（中国工程物理研究院计算机应用研究所）；Faculty of Electronic and Information Engineering, Xi’an Jiaotong University（西安交通大学电子与信息工程学院）；School of Cyberspace Security, Beijing University of Posts and Telecommunications（北京邮电大学网络空间安全学院）
针对可信数据集蒸馏可阻断传统投毒后门的问题，本文提出 INK，以自然图像方差为触发器，在无训练数据访问权时实现可从教师迁移到学生的干净图像、干净标签攻击，蒸馏后 ASR 仍超 98%。
Stealthy Yet Effective: Distribution-Preserving Backdoor Attacks on Graph Classification
College of Intelligence and Computing, Tianjin University, Tianjin, China（天津大学智能与计算学院）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ), Shenzhen, China（广东人工智能与数字经济实验室（深圳））；College of Artificial Intelligence, Dalian Maritime University, Dalian, China（大连海事大学人工智能学院）
Exposing Functional Fusion: A New Class of Strategic Backdoor in Dynamic Prompt Architectures
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；State Key Laboratory of Cyberspace Security Defense（网络空间安全防御国家重点实验室）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）；PetroChina (Beijing) Digital Intelligent Research Institute Co., Ltd.（中石油北京数字智能研究院有限公司）
Deep learning has come a long way and has enjoyed an unprecedented success. Despite high accuracy, however, deep models are brittle and are easily fooled by imperceptible adversarial perturbations. In contrast to common inference-time attacks, Backdoor (\aka Trojan) attacks target the training phase of model construction, and are extremely difficult to combat since a) the model behaves normally on a pristine testing set and b) the augmented perturbations can be minute and may only affect few training samples. Here, I propose a new method to tell whether a model has been subject to a backdoor a…
Amazon Alexa（亚马逊 Alexa）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）
Adversarial machine learning has exposed several security hazards of neural models and has become an important research topic in recent times. Thus far, the concept of an "adversarial perturbation" has exclusively been used with reference to the input space referring to a small, imperceptible change which can cause a ML model to err. In this work we extend the idea of "adversarial perturbations" to the space of model weights, specifically to inject backdoors in trained DNNs, which exposes a security risk of using publicly available trained models. Here, injecting a backdoor refers to obtaining…
Continual (or "incremental") learning approaches are employed when additional knowledge or tasks need to be learned from subsequent batches or from streaming data. However these approaches are typically adversary agnostic, i.e., they do not consider the possibility of a malicious attack. In our prior work, we explored the vulnerabilities of Elastic Weight Consolidation (EWC) to the perceptible misinformation. We now explore the vulnerabilities of other regularization-based as well as generative replay-based continual learning algorithms, and also extend the attack to imperceptible misinformati…
BadCM: Invisible Backdoor Attack Against Cross-Modal Learning
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）计算机科学与技术学院）；Peng Cheng Laboratory（鹏城实验室）；School of Electronic and Information Engineering, Tongji University（同济大学电子与信息工程学院）；School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）
Does Few-shot Learning Suffer from Backdoor Attacks?
Institute of Information Engineering, CAS（中国科学院信息工程研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Nanyang Technological University（南洋理工大学）；University of Oxford（牛津大学）；National University of Singapore（新加坡国立大学）；Sun Yat-sen University（中山大学）
SNEAKDOOR: Stealthy Backdoor Attacks against Distribution Matching-based Dataset Condensation
School of Computer Science and Technology, Xi’an Jiaotong University（西安交通大学计算机科学与技术学院）；National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi’an Jiaotong University（西安交通大学人机混合增强智能全国重点实验室）
BDFirewall: Towards Effective and Expeditiously Black-Box Backdoor Defense in MLaaS
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；State Key Laboratory for Novel Software Technology, Nanjing University（南京大学新型软件技术国家重点实验室）；Yangzhou University（扬州大学）
