IndicSafe：评估南亚多语言大语言模型安全性的基准
IndicSafe: A Benchmark for Evaluating Multilingual LLM Safety in South Asia
AI总结：
本文提出IndicSafe基准，评估12种南亚语言中LLM的安全性，发现跨语言一致性仅12.8%，安全率波动超17%，揭示多语言LLM安全泛化缺口。
摘要：
随着大语言模型（LLM）在多语言环境中的部署，其在文化多样性和低资源语言中的安全性行为仍不明确。我们首次系统评估了12种印地语系语言中LLM的安全性，这些语言由超过12亿人使用，但在LLM训练数据中代表性不足。使用覆盖种姓、宗教、性别、健康和政治的6000个文化相关提示集，我们评估了10种领先LLM在翻译提示变体上的表现。我们的分析揭示了显著的安全漂移：跨语言一致性仅为12.8%，安全率波动超过17%。某些模型在低资源脚本中过度拒绝良性提示，在政治敏感话题上过度标记，而其他模型未能标记不安全生成。我们使用提示级熵、类别偏见分数和多语言一致性指数量化这些失败。我们的发现突显了多语言LLM在安全泛化方面的关键缺口，并表明安全对齐在不同语言中并不均匀转移。我们发布了IndicSafe，这是首个能够为印地语部署提供文化知情安全评估的基准，并倡导基于地区危害的语言意识对齐策略。
英文摘要：
As large language models (LLMs) are deployed in multilingual settings, their safety behavior in culturally diverse, low-resource languages remains poorly understood. We present the first systematic evaluation of LLM safety across 12 Indic languages, spoken by over 1.2 billion people but underrepresented in LLM training data. Using a dataset of 6,000 culturally grounded prompts spanning caste, religion, gender, health, and politics, we assess 10 leading LLMs on translated variants of the prompt. Our analysis reveals significant safety drift: cross-language agreement is just 12.8\%, and \texttt{SAFE} rate variance exceeds 17\% across languages. Some models over-refuse benign prompts in low-resource scripts, overflag politically sensitive topics, while others fail to flag unsafe generations. We quantify these failures using prompt-level entropy, category bias scores, and multilingual consistency indices. Our findings highlight critical safety generalization gaps in multilingual LLMs and show that safety alignment does not transfer evenly across languages. We release \textsc{IndicSafe}, the first benchmark to enable culturally informed safety evaluation for Indic deployments, and advocate for language-aware alignment strategies grounded in regional harms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
