Safety aligned Large Language Models (LLMs) are vulnerable to harmful fine-tuning attacks -- a few harmful data mixed in the fine-tuning dataset can break the LLMs's safety alignment. While several defenses have been proposed, our evaluation shows that existing defenses fail \textit{when some specific training hyper-parameters are chosen} -- a large learning rate or a large number of training epochs in the fine-tuning stage can easily invalidate the defense. To this end, we propose Antidote, a post-fine-tuning stage solution, which remains \textbf{\textit{agnostic to the training hyper-parameters in the fine-tuning stage}}. Antidote relies on the philosophy that by removing the harmful parameters, the harmful model can be recovered from the harmful behaviors, regardless of how those harmful parameters are formed in the fine-tuning stage. With this philosophy, we introduce a one-shot pruning stage after harmful fine-tuning to remove the harmful weights that are responsible for the generation of harmful content. Despite its embarrassing simplicity, empirical results show that Antidote can reduce harmful score while maintaining accuracy on downstream tasks. Code is available at https://github.com/git-disl/Antidote.
A Guardrail for Safety Preservation: When Safety-Sensitive Subspace Meets Harmful-Resistant Null-Space
School of Artificial Intelligence, Jilin University(吉林大学人工智能学院);King Abdullah University of Science and Technology(卡塔尔国王大学科学与技术研究院);University of Oxford(牛津大学)
Behind Harmful Compliance: Behavioral and Mechanistic Divergence Across LLM Jailbreaks
Department of Computer Science(计算机科学系);Luddy School of Informatics, Computing, and Engineering(信息学、计算与工程学院);Indiana University Bloomington(印第安纳大学布卢明顿分校)
SPARD: Defending Harmful Fine-Tuning Attack via Safety Projection with Relevance-Diversity Data Selection
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology(香港科学与技术大学计算机科学与工程系);Department of Computer Science and Engineering, Southern University of Science and Technology(南方科技大学计算机科学与工程系);Department of Computer Science and Engineering, The Chinese University of Hong Kong(香港中文大学计算机科学与工程系);Platform and Content Group, Tencent(腾讯平台与内容组);Chinese Medicine Guangdong Laboratory(广东中医实验室)
Deactivating Refusal Triggers: Understanding and Mitigating Overrefusal in Safety Alignment
University of California, Santa Barbara(加州大学圣巴巴拉分校);Johns Hopkins University(约翰霍普金斯大学);Michigan State University(密歇根州立大学);University of Mississippi(密苏里州立大学)
Between a Rock and a Hard Place: The Tension Between Ethical Reasoning and Safety Alignment in LLMs
Department of Computer Science and Technology, Institute for AI, BNRist, Tsinghua University(计算机科学与技术系,人工智能研究院,BNRist,清华大学);IDG/McGovern Institute for Brain Research, Tsinghua University(IDG/麦克戈文脑科学研究院,清华大学);Chinese Institute for Brain Research (CIBR)(中国脑科学研究院(CIBR));Shanghai Jiao Tong University(上海交通大学);ByteDance(字节跳动)
Counterfactual Neighborhood Risk Fields: Evidence-Grounded Selective Safety for Vision-Language Models
Tongji University(同济大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);University of Electronic Science and Technology of China(电子科技大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
Beyond Visual Safety: Jailbreaking Multimodal Large Language Models for Harmful Image Generation via Semantic-Agnostic Inputs
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications(网络与交换技术国家重点实验室,北京邮电大学);School of Cyberspace Security, Beijing University of Posts and Telecommunications(网络安全学院,北京邮电大学)
Pharmacist: Safety Alignment Data Curation for Large Language Models against Harmful Fine-tuning
School of Computer Science and Engineering, South China University of Technology(华南理工大学计算机科学与工程学院);School of Computer Science at Georgia Institute of Technology(佐治亚理工学院计算机科学系);School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区网络科学与技术学院);Second Affiliated Hospital of Guangzhou University of Chinese Medicine(广州中医药大学第二附属医院);China and Pengcheng Laboratory(中 Pengcheng 实验室)
EVA: Editing for Versatile Alignment against Jailbreaks
ShanghaiTech University(上海科技大学);Sun Yat-sen University(中山大学);State Key Laboratory of Blockchain and Data Security(区块链与数据安全国家重点实验室);Tsinghua University(清华大学)
Safety Alignment via Constrained Knowledge Unlearning
Harbin Institute of Technology, Shenzhen, China(哈尔滨工业大学(深圳));University of Macau, China(澳门大学);Nankai University, China(南开大学);Zhejiang University, China(浙江大学);Peng Cheng Laboratory, China(鹏城实验室);Shaqra University, Saudi Arabia(沙克拉大学)
Preventing Safety Drift in Large Language Models via Coupled Weight and Activation Constraints
Hunan Normal University(湖南师范大学);The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences(认知与决策智能复杂系统重点实验室,中国科学院自动化研究所);National University of Defense Technology(国防科技大学)
The Safety Reminder: A Soft Prompt to Reactivate Delayed Safety Awareness in Vision-Language Models
School of Computer Science and Technology, Xi’an Jiaotong University(西安交通大学计算机科学与技术学院);School of Computer Science and Technology, University of Science and Technology of China(中国科学技术大学计算机科学与技术学院);School of Computing and Information Systems, Singapore Management University(新加坡管理学院计算与信息系统学院)
Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training
School of Data Science, The Chinese University of Hong Kong, Shenzhen, China(数据科学学院,香港中文大学(深圳));Tencent AI Lab(腾讯人工智能实验室);The Chinese University of Hong Kong(香港中文大学);Shenzhen Research Institute of Big Data, China(深圳大数据研究院)