Deep Neural Networks (DNNs) are known to be vulnerable to backdoor attacks. In Natural Language Processing (NLP), DNNs are often backdoored during the fine-tuning process of a large-scale Pre-trained Language Model (PLM) with poisoned samples. Although the clean weights of PLMs are readily available, existing methods have ignored this information in defending NLP models against backdoor attacks. In this work, we take the first step to exploit the pre-trained (unfine-tuned) weights to mitigate backdoors in fine-tuned language models. Specifically, we leverage the clean pre-trained weights via two complementary techniques: (1) a two-step Fine-mixing technique, which first mixes the backdoored weights (fine-tuned on poisoned data) with the pre-trained weights, then fine-tunes the mixed weights on a small subset of clean data; (2) an Embedding Purification (E-PUR) technique, which mitigates potential backdoors existing in the word embeddings. We compare Fine-mixing with typical backdoor mitigation methods on three single-sentence sentiment classification tasks and two sentence-pair classification tasks and show that it outperforms the baselines by a considerable margin in all scenarios. We also show that our E-PUR method can benefit existing mitigation methods. Our work establishes a simple but strong baseline defense for secure fine-tuned NLP models against backdoor attacks.
Merging Triggers, Breaking Backdoors: Defensive Poisoning for Instruction-Tuned Language Models
Graduate School of Artificial Intelligence, POSTECH, Republic of Korea（POSTECH人工智能研究生院，韩国）；Department of Computer Science and Engineering, POSTECH, Republic of Korea（POSTECH计算机科学与工程系，韩国）
DualEdit: Mitigating Safety Fallback in LLM Backdoor Editing via Affirmation-Refusal Regulation
University of Science and Technology of China（中国科学技术大学）；National University of Singapore（新加坡国立大学）；Singapore Management University（新加坡管理大学）；MoE Key Lab of BIPC, University of Science and Technology of China（中国科学技术大学信息与通信技术国家重点实验室）
Fudan University（复旦大学）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
Large pre-trained language models (PLMs) have proven to be a crucial component of modern natural language processing systems. PLMs typically need to be fine-tuned on task-specific downstream datasets, which makes it hard to claim the ownership of PLMs and protect the developer's intellectual property due to the catastrophic forgetting phenomenon. We show that PLMs can be watermarked with a multi-task learning framework by embedding backdoors triggered by specific inputs defined by the owners, and those watermarks are hard to remove even though the watermarked PLMs are fine-tuned on multiple do…
Test-time Backdoor Mitigation for Black-Box Large Language Models with Defensive Demonstrations
University of Southern California（南加州大学）；Harvard University（哈佛大学）；University of California, Davis（加州大学戴维斯分校）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）
Detecting Stealthy Backdoor Samples based on Intra-class Distance for Large Language Models
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing（未来区块链与隐私计算先进创新中心）；Institute of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
Mitigating Fine-tuning based Jailbreak Attack with Backdoor Enhanced Safety Alignment
University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；University of Michigan-Ann Arbor（密歇根大学安娜堡分校）；Princeton University（普林斯顿大学）；University of California, Davis（加州大学戴维斯分校）；University of Chicago（芝加哥大学）
At present, backdoor attacks attract attention as they do great harm to deep learning models. The adversary poisons the training data making the model being injected with a backdoor after being trained unconsciously by victims using the poisoned dataset. In the field of text, however, existing works do not provide sufficient defense against backdoor attacks. In this paper, we propose a Noise-augmented Contrastive Learning (NCL) framework to defend against textual backdoor attacks when training models with untrustworthy data. With the aim of mitigating the mapping between triggers and the targe…
\textbf{P}re-\textbf{T}rained \textbf{M}odel\textbf{s} have been widely applied and recently proved vulnerable under backdoor attacks: the released pre-trained weights can be maliciously poisoned with certain triggers. When the triggers are activated, even the fine-tuned model will predict pre-defined labels, causing a security threat. These backdoors generated by the poisoning methods can be erased by changing hyper-parameters during fine-tuning or detected by finding the triggers. In this paper, we propose a stronger weight-poisoning attack method that introduces a layerwise weight poisoning…
Tsinghua University（清华大学）；Huazhong University of Science and Technology（华中科技大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Textual backdoor attacks are a kind of practical threat to NLP systems. By injecting a backdoor in the training phase, the adversary could control model predictions via predefined triggers. As various attack and defense models have been proposed, it is of great significance to perform rigorous evaluations. However, we highlight two issues in previous backdoor learning evaluations: (1) The differences between real-world scenarios (e.g. releasing poisoned datasets or models) are neglected, and we argue that each scenario has its own constraints and concerns, thus requires specific evaluation pro…
Defending Against Weight-Poisoning Backdoor Attacks for Parameter-Efficient Fine-Tuning
Nanyang Technological University（南洋理工大学）；Hong Kong University of Science and Technology（香港科技大学）；Jinan University（暨南大学）；Sony Research（索尼研究院）；Zhejiang University（浙江大学）；Beijing Institute of Technology（北京理工大学）
Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attacks
Tsinghua University（清华大学）；Institute for Artificial Intelligence, Tsinghua University（清华大学人工智能研究院）；State Key Lab on Intelligent Technology and Systems, Tsinghua University（清华大学智能技术与系统国家重点实验室）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）
SCOUT: A Defense Against Data Poisoning Attacks in Fine-Tuned Language Models
Department of Computer and Information Sciences, Fordham University（福特汉姆大学计算机与信息科学系）；Department of Electrical Engineering, Zhejiang University（浙江大学电子工程系）；Department of Systems Engineering, City University of Hong Kong（香港城市大学系统工程系）；Department of Electrical and Computer Engineering, University of Michigan-Dearborn（密歇根大学迪尔伯恩分校电气与计算机工程系）
