arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.19103cs.CRcs.LG
收藏

针对GPT-4o的语音越狱攻击

Voice Jailbreak Attacks Against GPT-4o

CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全中心)

Xinyue Shen, Yixin Wu, Michael Backes, Yang Zhang

AI总结:

针对GPT-4o语音模式,提出基于人性化虚构故事叙述的VoiceJailbreak攻击,将平均攻击成功率从0.033提升至0.778,揭示语音交互新风险。

摘要:

近年来,人工智能助手的概念已从科幻小说演变为现实应用。GPT-4o作为最新的跨音频、视觉和文本的多模态大语言模型(MLLM),通过实现更自然的人机交互,进一步模糊了虚构与现实之间的界限。然而,GPT-4o语音模式的出现也可能引入新的攻击面。在本文中,我们首次对针对GPT-4o语音模式的越狱攻击进行了系统性测量。我们表明,当直接将禁止性问题及文本越狱提示迁移到语音模式时,GPT-4o表现出良好的抵抗能力。这种抵抗主要归因于GPT-4o的内部安全机制以及将文本越狱提示适配到语音模式的难度。受GPT-4o类人行为的启发,我们提出了VoiceJailbreak,一种新颖的语音越狱攻击,它使GPT-4o人性化,并试图通过虚构故事叙述(设定、角色和情节)来说服它。VoiceJailbreak能够生成简单、可听见且有效的越狱提示,在六种禁止场景中,平均攻击成功率(ASR)从0.033显著提升至0.778。我们还进行了大量实验,以探索交互步骤、虚构写作的关键元素以及不同语言对VoiceJailbreak有效性的影响,并通过先进的虚构写作技术进一步增强攻击性能。我们希望我们的研究能帮助研究社区构建更安全、更规范的MLLM。

英文摘要:

Recently, the concept of artificial assistants has evolved from science fiction into real-world applications. GPT-4o, the newest multimodal large language model (MLLM) across audio, vision, and text, has further blurred the line between fiction and reality by enabling more natural human-computer interactions. However, the advent of GPT-4o's voice mode may also introduce a new attack surface. In this paper, we present the first systematic measurement of jailbreak attacks against the voice mode of GPT-4o. We show that GPT-4o demonstrates good resistance to forbidden questions and text jailbreak prompts when directly transferring them to voice mode. This resistance is primarily due to GPT-4o's internal safeguards and the difficulty of adapting text jailbreak prompts to voice mode. Inspired by GPT-4o's human-like behaviors, we propose VoiceJailbreak, a novel voice jailbreak attack that humanizes GPT-4o and attempts to persuade it through fictional storytelling (setting, character, and plot). VoiceJailbreak is capable of generating simple, audible, yet effective jailbreak prompts, which significantly increases the average attack success rate (ASR) from 0.033 to 0.778 in six forbidden scenarios. We also conduct extensive experiments to explore the impacts of interaction steps, key elements of fictional writing, and different languages on VoiceJailbreak's effectiveness and further enhance the attack performance with advanced fictional writing techniques. We hope our study can assist the research community in building more secure and well-regulated MLLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑