JailbreakEval：一个用于评估针对大语言模型的越狱尝试的集成工具包
JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Models
AI总结：
本文系统分析了近90项越狱研究中的评估方法，提出评估器分类法，并开发了集成工具包JailbreakEval，以简化越狱评估流程并推动社区形成统一评估标准。
摘要：
越狱攻击诱导大语言模型（LLMs）生成有害响应，构成严重的滥用威胁。尽管关于越狱攻击与防御的研究正在兴起，但在越狱评估方面尚无共识，即评估LLM响应有害性的方法多种多样。每种方法都有其自身的优缺点，影响其与人类价值观的一致性，以及时间和财务成本。这种多样性给研究人员选择合适的评估方法以及比较不同的攻击和防御带来了挑战。在本文中，我们对越狱评估方法进行了全面分析，借鉴了2023年5月至2024年4月间发表的近90项越狱研究。我们的研究引入了一个系统的越狱评估器分类法，深入洞察其优缺点及其适应现状。为了辅助进一步研究，我们提出了JailbreakEval，一个用于评估越狱尝试的工具包。JailbreakEval开箱即用地包含多种评估器，使用户能够通过单个命令或自定义评估工作流获得结果。总之，我们认为JailbreakEval是一个催化剂，它简化了越狱研究中的评估过程，并在社区内培育了一个包容性的越狱评估标准。
英文摘要：
Jailbreak attacks induce Large Language Models (LLMs) to generate harmful responses, posing severe misuse threats. Though research on jailbreak attacks and defenses is emerging, there is no consensus on evaluating jailbreaks, i.e., the methods to assess the harmfulness of an LLM's response are varied. Each approach has its own set of strengths and weaknesses, impacting their alignment with human values, as well as the time and financial cost. This diversity challenges researchers in choosing suitable evaluation methods and comparing different attacks and defenses. In this paper, we conduct a comprehensive analysis of jailbreak evaluation methodologies, drawing from nearly 90 jailbreak research published between May 2023 and April 2024. Our study introduces a systematic taxonomy of jailbreak evaluators, offering indepth insights into their strengths and weaknesses, along with the current status of their adaptation. To aid further research, we propose JailbreakEval, a toolkit for evaluating jailbreak attempts. JailbreakEval includes various evaluators out-of-the-box, enabling users to obtain results with a single command or customized evaluation workflows. In summary, we regard JailbreakEval to be a catalyst that simplifies the evaluation process in jailbreak research and fosters an inclusive standard for jailbreak evaluation within the community.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
