arXiv 2508.200382025-09-04cs.CL

有备无患：预合成类越狱指令以增强LLM对潜在攻击的安全防护栏

Forewarned is Forearmed: Pre-Synthesizing Jailbreak-like Instructions to Enhance LLM Safety Guardrail to Potential Attacks

Sheng Liu Media Synthesis and Forensics Lab, Institute of Computing Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences（媒体合成与取证实验室，计算技术研究所，中国科学院，中国科学院大学）；Qiang Sheng Media Synthesis and Forensics Lab, Institute of Computing Technology, Chinese Academy of Sciences（媒体合成与取证实验室，计算技术研究所，中国科学院）；Danding Wang Media Synthesis and Forensics Lab, Institute of Computing Technology, Chinese Academy of Sciences（媒体合成与取证实验室，计算技术研究所，中国科学院）；Yang Li Media Synthesis and Forensics Lab, Institute of Computing Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences（媒体合成与取证实验室，计算技术研究所，中国科学院，中国科学院大学）；Guang Yang Zhongguancun Laboratory（中关村实验室）；Juan Cao Media Synthesis and Forensics Lab, Institute of Computing Technology, Chinese Academy of Sciences（媒体合成与取证实验室，计算技术研究所，中国科学院）

Sheng Liu, Qiang Sheng, Danding Wang, Yang Li, Guang Yang, Juan Cao

备注：: EMNLP 2025 findings

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对大语言模型易受越狱攻击、训练数据与真实攻击分布不匹配的问题，提出IMAGINE合成框架，通过嵌入空间分析和迭代优化生成类越狱指令以增强安全对齐语料，在多个主流LLM上显著降低攻击成功率且不影响效用。

摘要：

尽管在提升大语言模型（LLM）拒绝回答恶意指令的能力方面已有进展，当前广泛使用的LLM仍易受越狱攻击——攻击者会生成分布与安全对齐语料不同的指令。新型攻击暴露出LLM无法识别未见过的恶意指令的问题，凸显出训练数据与真实世界攻击之间存在严重的分布不匹配，迫使开发者陷入被动补丁修复的循环。为解决这一挑战，我们提出IMAGINE，这是一个利用嵌入空间分布分析生成类越狱指令的合成框架。该方法有效填补了真实越狱模式与安全对齐语料之间的分布差距。IMAGINE遵循迭代优化流程，在各迭代间动态演化文本生成分布，从而通过合成数据样本扩大安全对齐数据分布的覆盖范围。基于经IMAGINE增强的安全对齐语料，我们的框架在Qwen2.5、Llama3.1和Llama3.2上实现了攻击成功率的显著下降，且未损害其效用。

英文摘要：

Despite advances in improving large language model (LLM) to refuse to answer malicious instructions, widely used LLMs remain vulnerable to jailbreak attacks where attackers generate instructions with distributions differing from safety alignment corpora. New attacks expose LLMs' inability to recognize unseen malicious instructions, highlighting a critical distributional mismatch between training data and real-world attacks that forces developers into reactive patching cycles. To tackle this challenge, we propose IMAGINE, a synthesis framework that leverages embedding space distribution analysis to generate jailbreak-like instructions. This approach effectively fills the distributional gap between authentic jailbreak patterns and safety alignment corpora. IMAGINE follows an iterative optimization process that dynamically evolves text generation distributions across iterations, thereby augmenting the coverage of safety alignment data distributions through synthesized data examples. Based on the safety-aligned corpus enhanced through IMAGINE, our framework demonstrates significant decreases in attack success rate on Qwen2.5, Llama3.1, and Llama3.2 without compromising their utility.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。