We investigate security concerns of the emergent instruction tuning paradigm, that models are trained on crowdsourced datasets with task instructions to achieve superior performance. Our studies demonstrate that an attacker can inject backdoors by issuing very few malicious instructions (~1000 tokens) and control model behavior through data poisoning, without even the need to modify data instances or labels themselves. Through such instruction attacks, the attacker can achieve over 90% attack success rate across four commonly used NLP datasets. As an empirical study on instruction attacks, we systematically evaluated unique perspectives of instruction attacks, such as poison transfer where poisoned models can transfer to 15 diverse generative datasets in a zero-shot manner; instruction transfer where attackers can directly apply poisoned instruction on many other datasets; and poison resistance to continual finetuning. Lastly, we show that RLHF and clean demonstrations might mitigate such backdoors to some degree. These findings highlight the need for more robust defenses against poisoning attacks in instruction-tuning models and underscore the importance of ensuring data quality in instruction crowdsourcing.
Merging Triggers, Breaking Backdoors: Defensive Poisoning for Instruction-Tuned Language Models
Graduate School of Artificial Intelligence, POSTECH, Republic of Korea（POSTECH人工智能研究生院，韩国）；Department of Computer Science and Engineering, POSTECH, Republic of Korea（POSTECH计算机科学与工程系，韩国）
A Survey of Recent Backdoor Attacks and Defenses in Large Language Models
Nanyang Technological University（南洋理工大学）；Beijing Institute of Technology（北京理工大学）；University of St Andrews（圣安德鲁斯大学）；Zhejiang University（浙江大学）；Shanghai AI Lab（上海人工智能实验室）
School of Software and Microelectronics, Peking University（北京大学软件与微电子学院）；College of Cyber Science, Nankai University（南开大学网络空间安全学院）；The University of Western Australia（西澳大学）
Simulate and Eliminate: Revoke Backdoors for Generative Large Language Models
The Hong Kong University of Science and Technology（香港科技大学）；National University of Singapore（新加坡国立大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））
ProjLens: Unveiling the Role of Projectors in Multimodal Model Safety
University of Science and Technology of China（中国科学技术大学）；Beijing University of Aeronautics and Astronautics（北京航空航天大学）；Nanyang Technological University（南洋理工大学）；Southwest Jiaotong University（西南交通大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
MMJailBench: A Factorized Benchmark for Disentangling Multimodal Jailbreak Vulnerabilities
Tongji University（同济大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
TCAP: Tri-Component Attention Profiling for Unsupervised Backdoor Detection in MLLM Fine-Tuning
School of Control Science and Engineering, Shandong University, Jinan, China（控制科学与工程学院，山东大学，济南，中国）；Key Laboratory of Industrial Intelligent Systems, Shandong Province, China（山东省工业智能系统重点实验室，中国）
Test-time Backdoor Mitigation for Black-Box Large Language Models with Defensive Demonstrations
University of Southern California（南加州大学）；Harvard University（哈佛大学）；University of California, Davis（加州大学戴维斯分校）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）
Defending Against Weight-Poisoning Backdoor Attacks for Parameter-Efficient Fine-Tuning
Nanyang Technological University（南洋理工大学）；Hong Kong University of Science and Technology（香港科技大学）；Jinan University（暨南大学）；Sony Research（索尼研究院）；Zhejiang University（浙江大学）；Beijing Institute of Technology（北京理工大学）
Backdoor attack introduces artificial vulnerabilities into the model by poisoning a subset of the training data via injecting triggers and modifying labels. Various trigger design strategies have been explored to attack text classifiers, however, defending such attacks remains an open problem. In this work, we propose BFClass, a novel efficient backdoor-free training framework for text classification. The backbone of BFClass is a pre-trained discriminator that predicts whether each token in the corrupted input was replaced by a masked language model. To identify triggers, we utilize this discr…
