arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.21789cs.CLcs.AIcs.CY
收藏

以修正为幌子:框架与文化适应如何削弱大型语言模型(LLM)在真实刑事案件上的安全性

In the Guise of Correction: How Framing and Cultural Adaptation Erode LLM Safety on Real-World Criminal Cases

The University of Manchester(曼彻斯特大学);ELLIS Manchester(ELLIS曼彻斯特中心);Islamic University of Technology(伊斯兰技术大学);ELLIS Institute Finland(ELLIS芬兰研究所);University of Turku(图尔库大学)

Mohsinul Kabir, Md. Tariquzzaman, Karam S. R. Elradie, Polydoros Giannouris, Yuechen Jiang, Shaoxiong Ji, Sophia Ananiadou

AI总结:

该研究推出源自真实案件的GUISE基准,发现语言框架、文化适配会削弱LLM在刑事案件上的安全性,为强化跨语言文化的拒绝对齐提供资源并发布代码数据。

摘要:

现有安全基准大多依赖分类体系松散的合成提示,对严重犯罪类别缺乏针对性分析。我们推出GUISE——一个源自真实案件的犯罪协助场景基准,依据联合国毒品和犯罪问题办公室(UNODC)的《国际犯罪统计分类》(ICCS)分类体系组织。每个场景以三种语言框架呈现,并在五种不同文化中进行评估。采用我们提出的两层评估框架(同时判断危害性与操作提升度),研究发现:(i)仅框架本身就会导致不安全合规,平均不安全合规率从直接请求下的28.9%升至寻求修正时的66.4%(峰值达97.3%;p<0.05),因为修正模型会故意提供可执行的犯罪步骤;(ii)在所有四种非英语语言中,文化适配提示比相同请求的字面翻译引发更多不安全合规(+7至+18个百分点),这是基于翻译的基准所遗漏的风险;(iii)安全性最低的模型的不安全回应带有强烈的盎格鲁中心主义特征,且这种泄露程度随回应危害性单调递增(跨语言ρ=0.46至0.63,所有p<0.001)。GUISE为强化跨语言和跨文化的拒绝对齐提供了基于真实场景的资源与细粒度诊断,我们发布代码和数据以推动未来研究。

英文摘要:

Existing safety benchmarks rely largely on synthetic prompts with loosely grounded taxonomies and offer little focused analysis of severe crime categories. We introduce GUISE, a benchmark of criminal-assistance scenarios sourced from real-world cases and organised under a taxonomy from the UNODC International Classification of Crime for Statistical Purposes (ICCS). Each scenario is rendered under three linguistic framings and evaluated across five distinct cultures. Using our proposed two-tier evaluation framework, which judges both harmfulness and operational uplift, we find that: (i) framing alone drives unsafe compliance, with the mean unsafe compliance rate rising from 28.9% under direct requests to 66.4% under correction seeking (peaking at 97.3%; p < 0.05), as models correcting deliberately distorted details supply executable criminal steps; (ii) culturally adapted prompts elicit significantly more unsafe compliance than literal translations of the same requests in all four non-English languages (+7 to +18 points), a risk that translation-based benchmarks miss; and (iii) unsafe responses from the least-safe models carry strong Anglocentric registers, and the extent of this leakage increases monotonically with response harmfulness ($ρ$ = 0.46 to 0.63 across languages, all p < 0.001). GUISE offers a grounded resource and fine-grained diagnostics for strengthening refusal alignment across languages and cultures. We publish our code and data to facilitate future research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑