Harmful fine-tuning attack poses a serious threat to the online fine-tuning service. Vaccine, a recent alignment-stage defense, applies uniform perturbation to all layers of embedding to make the model robust to the simulated embedding drift. However, applying layer-wise uniform perturbation may lead to excess perturbations for some particular safety-irrelevant layers, resulting in defense performance degradation and unnecessary memory consumption. To address this limitation, we propose Targeted Vaccine (T-Vaccine), a memory-efficient safety alignment method that applies perturbation to only selected layers of the model. T-Vaccine follows two core steps: First, it uses gradient norm as a statistical metric to identify the safety-critical layers. Second, instead of applying uniform perturbation across all layers, T-Vaccine only applies perturbation to the safety-critical layers while keeping other layers frozen during training. Results show that T-Vaccine outperforms Vaccine in terms of both defense effectiveness and resource efficiency. Comparison with other defense baselines, e.g., RepNoise and TAR also demonstrate the superiority of T-Vaccine. Notably, T-Vaccine is the first defense that can address harmful fine-tuning issues for a 7B pre-trained models trained on consumer GPUs with limited memory (e.g., RTX 4090). Our code is available at https://github.com/Lslland/T-Vaccine.
Pharmacist: Safety Alignment Data Curation for Large Language Models against Harmful Fine-tuning
School of Computer Science and Engineering, South China University of Technology(华南理工大学计算机科学与工程学院);School of Computer Science at Georgia Institute of Technology(佐治亚理工学院计算机科学系);School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区网络科学与技术学院);Second Affiliated Hospital of Guangzhou University of Chinese Medicine(广州中医药大学第二附属医院);China and Pengcheng Laboratory(中 Pengcheng 实验室)
Reasoning-targeted Jailbreak Attacks on Large Reasoning Models via Semantic Triggers and Psychological Framing
College of Intelligence and Computing(智能与计算学院);School of New Media and Communication(新媒体与传播学院);Shanghai Key Laboratory of Data Science(上海数据科学 key laboratory)
Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs
Georgia Institute of Technology(佐治亚理工学院);University of Bristol(布里斯托大学);University of Maryland(马里兰大学);University College London(伦敦大学学院);MATS;Astra;MIT CSAIL(麻省理工学院计算机科学与人工智能实验室)
PrisonBreak: Jailbreaking Large Language Models with at Most Twenty-Five Targeted Bit-flips
Oregon State University(俄勒冈州立大学);University of British Columbia(不列颠哥伦比亚大学);University of Toronto(多伦多大学);George Mason University(乔治·梅森大学);Rutgers University(罗格斯大学);University of Texas at Arlington(德克萨斯大学阿灵顿分校)