基于响应的知识蒸馏用于多语言对抗预防无意中削弱了安全性
Response-Based Knowledge Distillation for Multilingual Jailbreak Prevention Unwittingly Compromises Safety
AI总结：
本文研究了基于响应的知识蒸馏在多语言对抗预防中的应用，发现标准微调会增加学生模型的对抗成功率，但去除边界拒绝可缓解安全性下降。
摘要：
大型语言模型（LLMs）在全球范围内日益普及，但其安全性对英语仍存在偏见，导致非英语语境下尤其是低资源语言的漏洞。本文提出了一种新的知识蒸馏（KD）应用，用于多语言对抗预防，通过LoRA将专有教师模型的拒绝行为蒸馏到三个开源学生模型中，使用约28,000个多语言对抗提示进行黑盒响应基于参数高效微调（PEFT）。在MultiJail基准测试中发现，标准微调教师的“安全”拒绝数据会无意中增加所有学生模型的对抗成功率（JSR），最高达16.6个百分点。实验揭示了蒸馏过程中对未见语言的泛化差异，不同基础模型结果各异。通过移除主要的安全性退化来源，即细微的“边界”拒绝，可以减轻或逆转学生模型的安全性下降，尽管推理性能（GSM8K）的减少仍存在。总体而言，本文的探索性研究突显了知识蒸馏作为多语言安全性对齐技术的挑战和潜力，为未来相关研究提供了基础。
英文摘要：
Large language models (LLMs) are increasingly deployed worldwide, yet their safety alignment remains predominantly English-centric. This allows for vulnerabilities in non-English contexts, especially with low-resource languages. We introduce a novel application of knowledge distillation (KD) in the context of multilingual jailbreak prevention, examining its efficacy. We distill the refusal behaviors of a proprietary teacher model (OpenAI o1-mini) with Low-Rank Adaptation (LoRA) into three open-source student models: Meta-Llama-3-8B-Instruct, Gemma-2-2B-IT, and Qwen3-8B, using ~28,000 multilingual jailbreak prompts from XSafety via black-box response-based, parameter-efficient fine-tuning (PEFT). Evaluation on the MultiJail benchmark reveals a counterintuitive behavior: standard fine-tuning on the teacher's ``safe'' refusal data inadvertently increases Jailbreak Success Rate (JSR) for all student models, up to 16.6 percentage points. Our experiments reveal a divergent generalization to unseen languages during distillation, with varying outcomes depending on the base model. By removing a primary source of safety degradation, nuanced `boundary' refusals, we mitigate or even reverse safety declines in student models, although reductions in reasoning performance (GSM8K) persist. Overall, our exploratory study highlights the challenges and potential of KD as a technique for multilingual safety alignment, offering a foundation for future research in this direction.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
