arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.16832cs.AIcs.CL
收藏

IndicJR:一种无裁判的南亚语言对抗鲁棒性基准

IndicJR: A Judge-Free Benchmark of Jailbreak Robustness in South Asian Languages

Oracle America Inc.(Oracle美国公司)

Priyaranjan Pattnayak, Sanchari Chowdhuri

AI总结:

IndicJR是一种无裁判的南亚语言对抗鲁棒性基准,揭示了多语言对抗攻击的模式和风险,尤其关注南亚用户的语言转换和罗马化问题。

摘要:

大型语言模型(LLMs)的安全对齐主要在英语中进行评估,并且具有合同约束,这使得多语言漏洞研究不足。我们引入了

英文摘要:

Safety alignment of large language models (LLMs) is mostly evaluated in English and contract-bound, leaving multilingual vulnerabilities understudied. We introduce \textbf{Indic Jailbreak Robustness (IJR)}, a judge-free benchmark for adversarial safety across 12 Indic and South Asian languages (2.1 Billion speakers), covering 45216 prompts in JSON (contract-bound) and Free (naturalistic) tracks. IJR reveals three patterns. (1) Contracts inflate refusals but do not stop jailbreaks: in JSON, LLaMA and Sarvam exceed 0.92 JSR, and in Free all models reach 1.0 with refusals collapsing. (2) English to Indic attacks transfer strongly, with format wrappers often outperforming instruction wrappers. (3) Orthography matters: romanized or mixed inputs reduce JSR under JSON, with correlations to romanization share and tokenization (approx 0.28 to 0.32) indicating systematic effects. Human audits confirm detector reliability, and lite-to-full comparisons preserve conclusions. IJR offers a reproducible multilingual stress test revealing risks hidden by English-only, contract-focused evaluations, especially for South Asian users who frequently code-switch and romanize.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑