当LLM遇见DRL：通过DRL引导的搜索提升越狱效率
When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided Search
AI总结：
本文提出RLbreaker，一种基于深度强化学习的黑盒越狱攻击方法，将越狱建模为搜索问题，通过定制PPO算法引导搜索，显著提升了对SOTA LLM的攻击效率，并具备鲁棒性和跨模型迁移能力。
摘要：
近期研究开发了越狱攻击，通过构造越狱提示词来欺骗LLM，使其对有害问题作出回应。早期的越狱攻击需要访问模型内部信息或大量人工努力。更高级的攻击利用遗传算法进行自动化和黑盒攻击。然而，遗传算法的随机性显著限制了这些攻击的有效性。在本文中，我们提出了RLbreaker，一种由深度强化学习（DRL）驱动的黑盒越狱攻击。我们将越狱建模为一个搜索问题，并设计一个RL智能体来引导搜索，这比随机搜索（如遗传算法）更有效且随机性更小。具体而言，我们针对越狱问题设计了一个定制的DRL系统，包括新颖的奖励函数和定制的近端策略优化（PPO）算法。通过大量实验，我们证明RLbreaker在针对六个最先进的（SOTA）LLM的攻击中，比现有越狱攻击有效得多。我们还表明，RLbreaker对三种SOTA防御具有鲁棒性，并且其训练好的智能体可以在不同LLM之间迁移。我们通过全面的消融研究进一步验证了RLbreaker的关键设计选择。
英文摘要：
Recent studies developed jailbreaking attacks, which construct jailbreaking prompts to fool LLMs into responding to harmful questions. Early-stage jailbreaking attacks require access to model internals or significant human efforts. More advanced attacks utilize genetic algorithms for automatic and black-box attacks. However, the random nature of genetic algorithms significantly limits the effectiveness of these attacks. In this paper, we propose RLbreaker, a black-box jailbreaking attack driven by deep reinforcement learning (DRL). We model jailbreaking as a search problem and design an RL agent to guide the search, which is more effective and has less randomness than stochastic search, such as genetic algorithms. Specifically, we design a customized DRL system for the jailbreaking problem, including a novel reward function and a customized proximal policy optimization (PPO) algorithm. Through extensive experiments, we demonstrate that RLbreaker is much more effective than existing jailbreaking attacks against six state-of-the-art (SOTA) LLMs. We also show that RLbreaker is robust against three SOTA defenses and its trained agents can transfer across different LLMs. We further validate the key design choices of RLbreaker via a comprehensive ablation study.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
