跨东亚和东南亚语境的文化适应红队测试：方法论与比较分析
Culturally-Adapted Red-Teaming Across East and Southeast Asian Contexts: A Methodological and Comparative Analysis
AI总结：
针对大语言模型的多语言安全评估，通过构建直接翻译与文化适应数据集，发现文化适应提示的攻击成功率平均提升9.3个百分点，直接翻译低估风险，且文化深度评分显著低于文化适应版本，表明适应文化语境对有效评估至关重要。
摘要：
大语言模型的多语言安全评估主要依赖于将英文基准直接翻译成目标语言——这种方法转换了表面语言形式，但未能反映威胁场景、社会规范和法律法规中嵌入的文化语境。我们通过1:1种子匹配为四种语言——韩语、日语、泰语和高棉语——构建了配对的直接翻译和文化适应数据集，并比较了四个开源大语言模型的攻击成功率和文化真实感评分。文化适应提示在所有16种语言×模型组合中均产生正Delta-ASR（平均+9.3个百分点），且基于直接翻译的评估在48个类别×语言组合中有44个低估了风险。语言层面分析显示，威胁形式的分布在语言间具有异质性。文化真实感分析进一步表明，直接翻译的文化深度（C3）评分在所有四种语言中始终低于1.0（满分3.0，平均0.17），而文化适应评分高达2.51，表明直接翻译产生的输入与真实世界多文化环境中遇到的输入存在系统性差异。这些发现表明，将基准适应特定语言的文化语境——而非仅依赖语言翻译——对于有效的多语言大语言模型安全评估是必要的。
英文摘要：
Multilingual safety evaluation of large language models (LLMs) has predominantly relied on direct translation (DT) of English benchmarks into target languages - an approach that converts surface-level linguistic form while failing to reflect the cultural context embedded in threat scenarios, social norms, and legal frameworks. We construct paired DT and culturally-adapted (CA) datasets via 1:1 seed matching for four languages - Korean (KO), Japanese (JA), Thai (TH), and Khmer (KM) - and compare Attack Success Rate (ASR) and Cultural Realism scores across four open-source LLM. CA prompts yield Delta-ASR > 0 across all 16 language x model combinations (mean +9.3 pp), and DT-based evaluation underestimates risk in 44 of 48 category x language combinations. Language-level analysis reveals that the distribution of threat forms is heterogeneous across languages. Cultural Realism analysis further shows that DT Cultural Depth (C3) scores remain consistently below 1.0 out of 3.0 across all four languages (mean 0.17), whereas CA scores reach up to 2.51, indicating that direct translation produces inputs systematically divergent from those encountered in real-world multicultural settings. These findings demonstrate that adapting benchmarks to language-specific cultural contexts - rather than relying on linguistic translation alone - is necessary for valid multilingual LLM safety evaluation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
