TukaBench: 一个基于文化的非洲语言越狱基准
TukaBench: A Culturally Grounded Jailbreak Benchmark for African Languages
AI总结：
针对大型语言模型在非洲低资源语言上的安全评估缺失，提出TUKABENCH基准，通过四种设置（直接翻译、文化适应翻译、人工策划提示、代码切换提示）评估语言、文化背景和提示规避性对模型安全的影响，发现非洲语言提示降低拒绝率，并引入Deflection指标和人工验证以解决模型理解失败和评判可靠性问题。
摘要：
大型语言模型（LLMs）的安全评估仍然高度以英语为中心，导致低资源语言（LRLs），特别是非洲语言，严重缺乏探索。我们引入了TUKABENCH，一个针对七种非洲语言的越狱基准，它通过四种设置将JailbreakBench（JBB）扩展到直接翻译之外：JBB提示的人工翻译、适应非洲背景的英语提示后人工翻译、通过与GPT-5.2交互验证的人工策划提示，以及结合英语和非洲语言的代码切换提示，从而隔离语言、文化背景和提示规避性对模型安全的影响。在闭源和开源模型中，使用非洲语言提示相比英语减少了拒绝，其中文化适应的提示导致最少的拒绝。评估还揭示了两个结构性限制：模型理解失败和低资源语言中LLM作为评判者的可靠性降低。为了捕捉前者，我们在“拒绝”和“越狱”之外引入了“回避”；为了评估后者，我们通过人工标注验证输出，显示在低资源语言和较少支持的脚本中，评判者与人类的一致性下降。
英文摘要：
Safety evaluation of Large Language Models (LLMs) remains heavily English-centric, leaving Low-Resource Languages (LRLs), particularly African ones, critically underexplored. We introduce TUKABENCH, a jailbreak benchmark for seven African languages that extends JailbreakBench (JBB) beyond direct translation through four settings: human translation of JBB prompts, English adaptation to African contexts followed by human translation, human-curated prompts validated through interactions with GPT-5.2, and code-switched prompts combining English and African languages, isolating the effect of language, cultural grounding, and prompt evasiveness on model safety. Across closed and open models, prompting in African languages reduces refusal relative to English, with culturally adapted prompts leading to least refusal. The evaluation also surfaces two structural limitations: model comprehension failures and reduced LLM-as-a-judge reliability in LRLs. To capture the first, we introduce Deflection alongside Refused and Jailbroken; to assess the second, we validate outputs with human annotations, showing that judge-human agreement drops in lower-resource languages and less commonly supported scripts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
