LinguaSafe：面向大语言模型的全面多语言安全基准
LinguaSafe: A Comprehensive Multilingual Safety Benchmark for Large Language Models
AI总结：
本文提出 LinguaSafe，一个包含 12 种语言、45k 条数据的多语言安全基准，通过直接与间接评估及过度敏感性检测，揭示不同语言间安全对齐差异，并公开数据集与代码以推动多语言 LLM 安全研究。
摘要：
大语言模型（LLMs）在全球技术中的广泛采用和日益突出的地位，要求我们必须严格关注其在各种语言和文化背景下的安全性。现有的多语言安全评估缺乏全面的评测和多样化的数据，限制了其有效性，阻碍了稳健的多语言安全对齐的发展。为了填补这一关键空白，我们提出了 LinguaSafe，这是一个以对语言真实性的细致关注而精心构建的全面多语言安全基准。LinguaSafe 数据集包含 12 种语言的 45k 条数据，涵盖从匈牙利语到马来语等多种语言。该数据集通过结合翻译、创译和本地来源数据精心构建，满足了对 LLMs 进行多语言安全评估的关键需求，填补了在从匈牙利语到马来语等多种代表性不足语言中 LLM 安全评估的空白。LinguaSafe 提供了一个多维度和细粒度的评估框架，包括直接和间接安全评估，并进一步包含对过度敏感性的评估。安全性和有用性评估的结果在不同领域和不同语言之间差异显著，即使在资源水平相似的语言中也是如此。我们的基准提供了一套全面的指标用于深入的安全评估，强调了彻底评估 LLMs 多语言安全性以实现更平衡的安全对齐的至关重要性。我们的数据集和代码已公开发布，以促进多语言 LLM 安全领域的进一步研究。
英文摘要：
The widespread adoption and increasing prominence of large language models (LLMs) in global technologies necessitate a rigorous focus on ensuring their safety across a diverse range of linguistic and cultural contexts. The lack of a comprehensive evaluation and diverse data in existing multilingual safety evaluations for LLMs limits their effectiveness, hindering the development of robust multilingual safety alignment. To address this critical gap, we introduce LinguaSafe, a comprehensive multilingual safety benchmark crafted with meticulous attention to linguistic authenticity. The LinguaSafe dataset comprises 45k entries in 12 languages, ranging from Hungarian to Malay. Curated using a combination of translated, transcreated, and natively-sourced data, our dataset addresses the critical need for multilingual safety evaluations of LLMs, filling the void in the safety evaluation of LLMs across diverse under-represented languages from Hungarian to Malay. LinguaSafe presents a multidimensional and fine-grained evaluation framework, with direct and indirect safety assessments, including further evaluations for oversensitivity. The results of safety and helpfulness evaluations vary significantly across different domains and different languages, even in languages with similar resource levels. Our benchmark provides a comprehensive suite of metrics for in-depth safety evaluation, underscoring the critical importance of thoroughly assessing multilingual safety in LLMs to achieve more balanced safety alignment. Our dataset and code are released to the public to facilitate further research in the field of multilingual LLM safety.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
