多语言拒绝对齐：构建更安全的大型语言模型
Multilingual Refusal Alignment for Safer Large Language Models
AI总结：
本研究系统探究多语言对齐动态，通过引入覆盖12种欧洲语言的RefusEU数据集和DPO实验，发现仅用英语对齐不足以保障跨语言安全，而多语言训练可在不降低通用性能的前提下提升安全性。
摘要：
随着大型语言模型（LLMs）在全球范围部署，确保其在多种语言中的安全性和对齐变得至关重要。然而，安全行为在不同语言之间往往表现出不可预测的差异，这对一致且合乎道德的人工智能构成了重大挑战。在这项工作中，我们系统研究了多语言对齐的动态，探讨了单语言对齐是否能够跨语言迁移、训练过程中如何保持语言一致性，以及由此产生的与通用知识能力之间的权衡。我们引入了RefusEU，一个覆盖12种欧洲语言的新型拒绝对齐数据集，其中包括一个用于评估当前最先进模型的专用测试集。我们的受控直接偏好优化（DPO）实验提供了两个关键见解：仅在英语中对齐模型不足以确保跨语言安全性，即使对于相同的危害类别也是如此；而使用多语言数据集进行训练可以在不降低通用性能（通过Global MMLU基准衡量）的情况下提高安全性。
英文摘要：
As Large Language Models (LLMs) are deployed globally, ensuring their safety and alignment across multiple languages becomes paramount. However, safety behaviors often vary unpredictably between languages, posing significant challenges for consistent and ethical AI. In this work, we systematically investigate the dynamics of multilingual alignment, exploring whether single-language alignment transfers cross-lingually, how language consistency is preserved during training, and the resulting trade-offs with general knowledge capabilities. We introduce RefusEU, a novel refusal alignment dataset covering 12 European languages, including a dedicated test set for evaluating current state-of-the-art models. Our controlled Direct Preference Optimization (DPO) experiments provide two key insights: aligning models exclusively in English is insufficient to ensure cross-lingual safety, even for the same harm categories, whereas training on multilingual datasets can improve safety without degrading general performance, as measured by the Global MMLU benchmark.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
