arXiv 2505.122872025-05-18cs.CLcs.AI

重访巴别塔：闭源大语言模型上的多语言越狱提示

The Tower of Babel Revisited: Multilingual Jailbreak Prompts on Closed-Source Large Language Models

School of Electrical and Computer Engineering, University of Sydney（电子与计算机工程学院，悉尼大学）；The University of Tokyo, Japan（东京大学，日本）

Linghan Huang, Haolin Jin, Zhaoge Bi, Pengyue Yang, Peizhou Zhao, Taozhao Chen, Xiongfei Wu, Lei Ma, Huaming Chen

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

该研究针对闭源大语言模型的多语言越狱攻击问题，提出集成对抗框架，评估多款前沿闭源模型在中英双语32种攻击下的表现，发现中文提示攻击成功率更高、新型Two-Sides攻击最有效，为跨语言安全防御提供参考。

摘要：

大语言模型（LLM）已在各领域得到广泛应用，但仍易受对抗性提示注入攻击。现有越狱攻击与幻觉现象研究大多聚焦于开源模型，我们则针对多语言攻击场景下的闭源大语言模型前沿展开研究。我们提出了首个集成式对抗框架，该框架利用多种攻击技术，系统评估GPT-4o、DeepSeek-R1、Gemini-1.5-Pro与Qwen-Max等前沿专有模型。我们的评估覆盖英文与中文的六大类安全内容，在32种越狱攻击下生成了38400条回复。我们以攻击成功率（ASR）为量化指标，从提示设计、模型架构与语言环境三个维度评估性能。研究结果显示，Qwen-Max最易受攻击，而GPT-4o的防御能力最强。值得注意的是，中文提示的攻击成功率始终高于对应的英文提示，且我们提出的新型Two-Sides攻击技术在所有模型上均表现出最佳效果。本研究凸显了大语言模型亟需具备语言感知的对齐机制与稳健的跨语言防御能力，我们希望能激励研究者、开发者与政策制定者打造更稳健、更具包容性的AI系统。

英文摘要：

Large language models (LLMs) have seen widespread applications across various domains, yet remain vulnerable to adversarial prompt injections. While most existing research on jailbreak attacks and hallucination phenomena has focused primarily on open-source models, we investigate the frontier of closed-source LLMs under multilingual attack scenarios. We present a first-of-its-kind integrated adversarial framework that leverages diverse attack techniques to systematically evaluate frontier proprietary solutions, including GPT-4o, DeepSeek-R1, Gemini-1.5-Pro, and Qwen-Max. Our evaluation spans six categories of security contents in both English and Chinese, generating 38,400 responses across 32 types of jailbreak attacks. Attack success rate (ASR) is utilized as the quantitative metric to assess performance from three dimensions: prompt design, model architecture, and language environment. Our findings suggest that Qwen-Max is the most vulnerable, while GPT-4o shows the strongest defense. Notably, prompts in Chinese consistently yield higher ASRs than their English counterparts, and our novel Two-Sides attack technique proves to be the most effective across all models. This work highlights a dire need for language-aware alignment and robust cross-lingual defenses in LLMs, and we hope it will inspire researchers, developers, and policymakers toward more robust and inclusive AI systems.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。