arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.14853cs.CLcs.AIcs.CR
收藏

Atoxia:利用目标有害回答对大型语言模型进行红队测试

Atoxia: Red-teaming Large Language Models with Target Toxic Answers

Shenzhen Research Institute of Big Data(深圳大数据研究院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));ByteDance(字节跳动);Tencent AI Lab(腾讯AI实验室)

Yuhao Du, Zhuo Li, Pengyu Cheng, Xiang Wan, Anningzhe Gao

AI总结:

本文提出 Atoxia 红队方法,通过目标有害回答、对应查询与误导性开头,并以强化学习优化攻击成功率,可在开源模型及 GPT-4o 等黑盒模型中发现安全风险。

摘要:

尽管人工智能取得了重大进展,大型语言模型(LLM)仍面临生成安全方面的挑战。借助对抗性越狱提示,人们可以毫不费力地诱导 LLM 输出有害内容,从而造成意外的负面社会影响。这一脆弱性凸显出,在大规模应用之前,需要有稳健的 LLM 红队策略来识别并缓解此类风险。为检测特定类型的风险,我们提出了一种新颖的红队测试方法,该方法利用目标有害回答(Target Toxic Answers,Atoxia)攻击 LLM。给定一个特定的有害回答,Atoxia 会生成相应的用户查询和具有误导性的回答开头,以检查给定 LLM 的内部缺陷。所提出的攻击者在强化学习方案中进行训练,并将 LLM 输出目标回答的概率作为奖励。我们在 AdvBench 和 HH-Harmless 等多种红队测试基准上验证了方法的有效性。实证结果表明,Atoxia 不仅能够成功检测开源模型中的安全风险,也能检测 GPT-4o 等最先进黑盒模型中的安全风险。

英文摘要:

Despite the substantial advancements in artificial intelligence, large language models (LLMs) remain being challenged by generation safety. With adversarial jailbreaking prompts, one can effortlessly induce LLMs to output harmful content, causing unexpected negative social impacts. This vulnerability highlights the necessity for robust LLM red-teaming strategies to identify and mitigate such risks before large-scale application. To detect specific types of risks, we propose a novel red-teaming method that $\textbf{A}$ttacks LLMs with $\textbf{T}$arget $\textbf{Toxi}$c $\textbf{A}$nswers ($\textbf{Atoxia}$). Given a particular harmful answer, Atoxia generates a corresponding user query and a misleading answer opening to examine the internal defects of a given LLM. The proposed attacker is trained within a reinforcement learning scheme with the LLM outputting probability of the target answer as the reward. We verify the effectiveness of our method on various red-teaming benchmarks, such as AdvBench and HH-Harmless. The empirical results demonstrate that Atoxia can successfully detect safety risks in not only open-source models but also state-of-the-art black-box models such as GPT-4o.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑