Generative large language models (LLMs) have achieved state-of-the-art results on a wide range of tasks, yet they remain susceptible to backdoor attacks: carefully crafted triggers in the input can manipulate the model to produce adversary-specified outputs. While prior research has predominantly focused on backdoor risks in vision and classification settings, the vulnerability of LLMs in open-ended text generation remains underexplored. To fill this gap, we introduce BackdoorLLM (Our BackdoorLLM benchmark was awarded First Prize in the SafetyBench competition, https://www.mlsafety.org/safebench/winners, organized by the Center for AI Safety, https://safe.ai/.), the first comprehensive benchmark for systematically evaluating backdoor threats in text-generation LLMs. BackdoorLLM provides: (i) a unified repository of benchmarks with a standardized training and evaluation pipeline; (ii) a diverse suite of attack modalities, including data poisoning, weight poisoning, hidden-state manipulation, and chain-of-thought hijacking; (iii) over 200 experiments spanning 8 distinct attack strategies, 7 real-world scenarios, and 6 model architectures; (iv) key insights into the factors that govern backdoor effectiveness and failure modes in LLMs; and (v) a defense toolkit encompassing 7 representative mitigation techniques. Our code and datasets are available at https://github.com/bboylyg/BackdoorLLM. We will continuously incorporate emerging attack and defense methodologies to support the research in advancing the safety and reliability of LLMs.
Merging Triggers, Breaking Backdoors: Defensive Poisoning for Instruction-Tuned Language Models
Graduate School of Artificial Intelligence, POSTECH, Republic of Korea(POSTECH人工智能研究生院,韩国);Department of Computer Science and Engineering, POSTECH, Republic of Korea(POSTECH计算机科学与工程系,韩国)
Proactive Adversarial Defense: Harnessing Prompt Tuning in Vision-Language Models to Detect Unseen Backdoored Images
University of West Florida(西佛罗里达大学);Florida Institute For Human and Machine Cognition (IHMC)(佛罗里达人机认知研究所);Center for Cybersecurity, University of West Florida(西佛罗里达大学网络安全中心)
Defense Against LLM Backdoors using Critical Neuron Isolation Pruning
Huazhong University of Science and Technology(华中科技大学);Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Nanyang Technological University(南洋理工大学)
Enhancing Fine-Tuning Based Backdoor Defense with Sharpness-Aware Minimization
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shenzhen Research Institute of Big Data(深圳市大数据研究院);JD Explore Academy(京东探索研究院);Tencent AI Lab(腾讯AI实验室)
BehaviorGuard: Online Backdoor Defense for Deep Reinforcement Learning
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics(南京航空航天大学人工智能学院);School of Cybersecurity, Northwestern Polytechnical University(西北工业大学网络安全学院);School of Computer Science and Technology, Harbin Institute of Technology(哈尔滨工业大学计算机科学与技术学院);Department of Electronic and Electrical Engineering, University College London(伦敦大学学院电子与电气工程系)
Expose Backdoors on the Way: A Feature-Based Efficient Defense against Textual Backdoor Attacks
Peking University(北京大学)
自然语言处理(Natural Language Processing,NLP)模型易受后门攻击的攻击,这是NLP模型面临的一个新的威胁。现有的NLP模型在线后门防御方法只关注输入或输出层面的异常,仍然存在易受自适应攻击和计算代价高的问题。在这项工作中,我们采取的第一步,在中间特征水平上研究文本中毒样本的去隐藏,并提出了一种基于特征的有效的在线防御方法。通过对现有攻击方法的大量实验,我们发现中毒样本在中毒NLP模型的中间特征空间中远离干净样本。出于这一观察,我们设计了一个基于距离的异常分数(DAN)区分中毒的样本从干净的样本在功能级别。在情感分析和攻击检测任务上的实验证明了DAN的优越性,因为它在防御性能方面大大超过现有的在线防御方法,并且具有较低的推理成本。此外,我们还证明了DAN也能抵抗基于特征级正则化的自适应攻击。我们的代码可在https://github.com/lancopku/DAN上获得。
MARS: A Malignity-Aware Backdoor Defense in Federated Learning
Faculty of Data Science, City University of Macau(澳门城市大学数据科学学院);School of Computing, Australian National University(澳大利亚国立大学计算机学院);Ant Group(蚂蚁集团);School of Cyber Science and Engineering, Huazhong University of Science and Technology(华中科技大学网络安全工程学院);School of Computer Science and Technology, Huazhong University of Science and Technology(华中科技大学计算机科学与技术学院);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院);School of Information and Communication Technology, Griffith University(格里菲斯大学信息与通信技术学院)
Towards Backdoor Stealthiness in Model Parameter Space
Radboud University(拉德博德大学);Delft University of Technology(代尔夫特理工大学);Radboud University, Nijmegen, The Netherlands(拉德博德大学);University of Zagreb Faculty of Electrical Engineering and Computing(Zagreb 工程与计算学院)
SoK: The Last Line of Defense: On Backdoor Defense Evaluation
University of Bergen(卑尔根大学);Radboud University(拉德博德大学);Delft University of Technology(代尔夫特理工大学);University of Pavia(帕维亚大学);Ikerlan Research Center Radboud University(Ikerlan研究中心拉德博德大学)
Learning Normal Diffusion Dynamics for Backdoor Defense in Text-to-Image Models
Geely(吉利);Hunan University(湖南大学);East China Normal University(华东师范大学);University of Adelaide(阿德莱德大学);The Hong Kong University of Science and Technology(香港科技大学);China University of Petroleum (East China)(中国石油大学(华东))
Backdoor Vectors: a Task Arithmetic View on Backdoor Attacks and Defenses
Warsaw University of Technology(华沙技术大学);NASK National Research Institute(国家研究机构NASK);IDEAS NCBR;IDEAS Research Institute(IDEAS研究所);Computer Vision Center, Universitat Autonoma de Barcelona(巴塞罗那自治大学计算机视觉中心)
REFINE: Inversion-Free Backdoor Defense via Model Reprogramming
Zhejiang University(浙江大学);Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security(杭州高新区(滨江)区块链与数据安全研究院);Nanyang Technological University(南洋理工大学);IBM Research(IBM研究院)
Class-Conditional Neural Polarizer: A Lightweight and Effective Backdoor Defense by Purifying Poisoned Features
School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)数据科学学院);Shenzhen Key Laboratory of Crowd Intelligence Empowered Low-Carbon Energy Network(深圳市人群智能赋能低碳能源网络重点实验室)
Evolutionary Trigger Detection and Lightweight Model Repair Based Backdoor Defense
Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies(广东省新型安全智能技术重点实验室);School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳)计算机科学与技术学院);Peng Cheng Laboratory(鹏城实验室);School of Computer Science and Engineering, Southern University of Science and Technology(南方科技大学计算机科学与工程学院)
TED-LaST: Towards Robust Backdoor Defense Against Adaptive Attacks
Deakin University(德肯大学);Beijing Normal-Hong Kong Baptist University(北京师范大学-香港 Baptist大学);University of New South Wales(新南威尔士大学);Griffith University(格里菲斯大学)
Defending the Edge: Representative-Attention Defense against Backdoor Attacks in Federated Learning
Dept. of Computer Science Faculty of Engineering University of Manchester Manchester, United Kingdom;Dept. of Computer Science Faculty of Engineering University of Manchester \& MBZUAI Abu Dhabi, United Arab Emirates;Dept. of Computer Science Faculty of Engineering University of Manchester \& KU Leuven Manchester, United Kingdom