arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.17710cs.CRcs.AI
收藏

基于优化的提示注入攻击:针对LLM-as-a-Judge

Optimization-based Prompt Injection Attack to LLM-as-a-Judge

Huazhong University of Science and Technology(华中科技大学);University of Notre Dame(圣母大学);Lehigh University(莱文森大学);Duke University(杜克大学)

Jiawen Shi, Zenghui Yuan, Yinuo Liu, Yue Huang, Pan Zhou, Lichao Sun, Neil Zhenqiang Gong

AI总结:

针对LLM-as-a-Judge在多领域应用中的安全问题,提出基于优化的提示注入攻击JudgeDeceiver,通过构造序列注入候选响应实现定向选择,实验证明其比现有攻击更有效,且现有防御措施不足,需新防御策略。

摘要:

LLM-as-a-Judge利用大型语言模型(LLM)为给定问题从候选响应集中选择最佳响应,在LLM驱动的搜索、基于AI反馈的强化学习(RLAIF)和工具选择等领域应用广泛。本文提出JudgeDeceiver,一种针对LLM-as-a-Judge的基于优化的提示注入攻击,通过向攻击者控制的候选响应中注入精心构造的序列,使LLM-as-a-Judge在攻击者选定的问题中无论其他候选响应如何都选择该候选响应。具体而言,我们将寻找此类序列的问题建模为优化问题,并提出一种基于梯度的方法来近似求解。广泛评估表明,JudgeDeceiver具有高度有效性,相比手动构造注入序列的现有提示注入攻击以及扩展到该问题的越狱攻击,其效果显著更优。我们还通过LLM驱动的搜索、RLAIF和工具选择三个案例研究验证了JudgeDeceiver的有效性。此外,我们测试了已知答案检测、困惑度检测和窗口化困惑度检测等防御方法,结果显示这些防御措施不足,凸显了开发新防御策略的迫切需求。我们的实现可在以下仓库获取:https://github.com/ShiJiawenwen/JudgeDeceiver。

英文摘要:

LLM-as-a-Judge uses a large language model (LLM) to select the best response from a set of candidates for a given question. LLM-as-a-Judge has many applications such as LLM-powered search, reinforcement learning with AI feedback (RLAIF), and tool selection. In this work, we propose JudgeDeceiver, an optimization-based prompt injection attack to LLM-as-a-Judge. JudgeDeceiver injects a carefully crafted sequence into an attacker-controlled candidate response such that LLM-as-a-Judge selects the candidate response for an attacker-chosen question no matter what other candidate responses are. Specifically, we formulate finding such sequence as an optimization problem and propose a gradient based method to approximately solve it. Our extensive evaluation shows that JudgeDeceive is highly effective, and is much more effective than existing prompt injection attacks that manually craft the injected sequences and jailbreak attacks when extended to our problem. We also show the effectiveness of JudgeDeceiver in three case studies, i.e., LLM-powered search, RLAIF, and tool selection. Moreover, we consider defenses including known-answer detection, perplexity detection, and perplexity windowed detection. Our results show these defenses are insufficient, highlighting the urgent need for developing new defense strategies. Our implementation is available at this repository: https://github.com/ShiJiawenwen/JudgeDeceiver.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑