Current vision large language models (VLLMs) exhibit remarkable capabilities yet are prone to generate harmful content and are vulnerable to even the simplest jailbreaking attacks. Our initial analysis finds that this is due to the presence of harmful data during vision-language instruction fine-tuning, and that VLLM fine-tuning can cause forgetting of safety alignment previously learned by the underpinning LLM. To address this issue, we first curate a vision-language safe instruction-following dataset VLGuard covering various harmful categories. Our experiments demonstrate that integrating this dataset into standard vision-language fine-tuning or utilizing it for post-hoc fine-tuning effectively safety aligns VLLMs. This alignment is achieved with minimal impact on, or even enhancement of, the models' helpfulness. The versatility of our safety fine-tuning dataset makes it a valuable resource for safety-testing existing VLLMs, training new models or safeguarding pre-trained VLLMs. Empirical results demonstrate that fine-tuned VLLMs effectively reject unsafe instructions and substantially reduce the success rates of several black-box adversarial attacks, which approach zero in many cases. The code and dataset are available at https://github.com/ys-zong/VLGuard.
LookAhead Tuning: Safer Language Models via Partial Answer Previews
Zhejiang University(浙江大学);Zhejiang University - Ant Group Joint Laboratory of Knowledge Graph(浙江大学-蚂蚁集团知识图谱联合实验室);Ant Group(蚂蚁集团);National University of Singapore(新加坡国立大学)
Representation Bending for Large Language Model Safety
Seoul National University(首尔国立大学);Yonsei University(延世大学);AIM Intelligence(AIM智能);University of Michigan(密歇根大学);Stanford University(斯坦福大学);Amazon AWS(亚马逊AWS)
Antibody: Strengthening Defense Against Harmful Fine-Tuning for Large Language Models via Attenuating Harmful Gradient Influence
Department of Electrical and Computer Systems Engineering, Monash University, Australia(莫纳什大学电气与计算机系统工程系);Department of Data Science and AI, Monash University, Australia(莫纳什大学数据科学与人工智能系)
Rethinking Safety in LLM Fine-tuning: An Optimization Perspective
Microsoft Research(微软研究院);KAIST(韩国科学技术院);KAUST(韩国科学技术大学);Université de Montréal(蒙特利尔大学);Mila(蒙特利尔人工智能研究院);WhiteBox(WhiteBox公司);University of Oxford(牛津大学)
What Makes and Breaks Safety Fine-tuning? A Mechanistic Study
Five AI Ltd.(Five AI有限公司);University of Michigan(密歇根大学);Harvard University(哈佛大学);University of Oxford(牛津大学);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);University of Copenhagen(哥本哈根大学)
Safe Delta: Consistently Preserving Safety when Fine-Tuning LLMs on Diverse Datasets
Guangdong Provincial Key Laboratory of Brain-Inspired Intelligent Computation, Department of CSE, SUSTech(脑启发智能计算广东省重点实验室,计算机科学与工程系,华南理工大学);Department of CSE, HKUST(计算机科学与工程系,香港科技大学);Department of CSE, PolyU(计算机科学与工程系,PolyU);Department of CSE, SUSTech(计算机科学与工程系,华南理工大学);Huawei Technologies CO.(华为技术有限公司);College of Computing(计算学院)
SPARD: Defending Harmful Fine-Tuning Attack via Safety Projection with Relevance-Diversity Data Selection
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology(香港科学与技术大学计算机科学与工程系);Department of Computer Science and Engineering, Southern University of Science and Technology(南方科技大学计算机科学与工程系);Department of Computer Science and Engineering, The Chinese University of Hong Kong(香港中文大学计算机科学与工程系);Platform and Content Group, Tencent(腾讯平台与内容组);Chinese Medicine Guangdong Laboratory(广东中医实验室)
Towards Understanding the Fragility of Multilingual LLMs against Fine-Tuning Attacks
Meta;University of Pisa(比萨大学);University of Modena and Reggio Emilia(摩德纳与雷焦艾米利亚大学);Brown University(布朗大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
Towards Resilient Safety-driven Unlearning for Diffusion Models against Downstream Fine-tuning
Nanyang Technological University, Singapore(南洋理工大学,新加坡);Central South University, China(中南大学,中国);Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University, China(航空航天信息安全部门,教育部,武汉大学,中国);State Key Laboratory of Blockchain and Data Security, Zhejiang University, China(区块链与数据安全国家重点实验室,浙江大学,中国)