Schützen: 在保加利亚语和德语语境中评估LLM安全性
Schützen: Evaluating LLM Safety in Bulgarian and German Contexts
AI总结：
针对现有安全评估数据集以英语和中文为主的问题，构建了覆盖低资源语言保加利亚语和高资源语言德语的Schützen安全数据集，实验揭示多语言LLM在安全行为上的显著跨语言差异，强调了区域特定评估资源的必要性。
摘要：
大型语言模型越来越多地部署在专业领域，带来了难以预测的风险，包括生成有害或不尊重的内容。尽管在开发安全评估数据集方面取得了实质性进展，但现有资源仍然 overwhelmingly 以英语和中文为中心。这种限制在评估共享社会文化、法律和伦理背景下的语言时尤为明显。为了解决这一差距，我们引入了Schützen：一个德语-保加利亚语安全数据集，旨在评估模型在风险下的可回答性，涵盖低资源语言（保加利亚语）和高资源语言（德语）。使用多语言和特定语言LLMs的实验揭示了安全行为中显著的跨语言差异，强调了需要定制的、特定区域的评估资源，以支持在德国和保加利亚负责任地部署LLMs。数据集和代码可在以下网址获取：https://this URL。警告：本文包含可能具有冒犯性、有害性或偏见性的示例。
英文摘要：
Large language models are increasingly deployed across professional domains, bringing hard-to-predict risks, including the generation of harmful or disrespectful content. Although substantial progress has been made in developing safety evaluation datasets, existing resources remain overwhelmingly English- and Chinese-centric. This limitation is particularly pronounced when evaluating languages that operate within shared sociocultural, legal, and ethical contexts. To address this gap, we introduce Schützen: a German--Bulgarian safety dataset designed to assess model answerability under risk, covering both a low-resource language (Bulgarian) and a high-resource language (German). Experiments with multilingual and language-specific LLMs reveal pronounced cross-language differences in safety behavior, highlighting the necessity of tailored, region-specific evaluation resources to support the responsible deployment of LLMs in Germany and Bulgaria. Datasets and code are available at https://github.com/xnlp-lab/Schutzen. Warning: this paper contains examples that may be offensive, harmful, or biased.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
