arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.13494cs.CLcs.CR
收藏

GradSafe:通过安全关键梯度分析检测 LLM 的越狱提示

GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysis

HKUST(香港科技大学);University of Louisville(路易斯维尔大学);Duke University(杜克大学)

Yueqi Xie, Minghong Fang, Renjie Pi, Neil Gong

AI总结:

提出GradSafe,通过分析LLM安全关键参数的梯度模式检测越狱提示,无需额外训练即优于微调的Llama Guard。

摘要:

大型语言模型(LLMs)面临越狱提示的威胁。现有的越狱提示检测方法主要是在线内容审核 API 或微调的 LLM。然而,这些策略通常需要大量且耗费资源的数据收集和训练过程。在本研究中,我们提出 GradSafe,通过审查 LLM 中安全关键参数的梯度来有效检测越狱提示。我们的方法基于一个关键观察:越狱提示与顺从响应配对时,LLM 的损失梯度在某些安全关键参数上表现出相似的模式。相比之下,安全提示则导致不同的梯度模式。基于这一观察,GradSafe 分析来自提示(与顺从响应配对)的梯度,以准确检测越狱提示。我们表明,应用于 Llama-2 而无需进一步训练的 GradSafe,在检测越狱提示方面优于 Llama Guard,尽管后者使用大型数据集进行了广泛微调。这种卓越的性能在零样本和适应场景中均保持一致,正如我们在 ToxicChat 和 XSTest 上的评估所证明的那样。源代码可在 https://github.com/xyq7/GradSafe 获取。

英文摘要:

Large Language Models (LLMs) face threats from jailbreak prompts. Existing methods for detecting jailbreak prompts are primarily online moderation APIs or finetuned LLMs. These strategies, however, often require extensive and resource-intensive data collection and training processes. In this study, we propose GradSafe, which effectively detects jailbreak prompts by scrutinizing the gradients of safety-critical parameters in LLMs. Our method is grounded in a pivotal observation: the gradients of an LLM's loss for jailbreak prompts paired with compliance response exhibit similar patterns on certain safety-critical parameters. In contrast, safe prompts lead to different gradient patterns. Building on this observation, GradSafe analyzes the gradients from prompts (paired with compliance responses) to accurately detect jailbreak prompts. We show that GradSafe, applied to Llama-2 without further training, outperforms Llama Guard, despite its extensive finetuning with a large dataset, in detecting jailbreak prompts. This superior performance is consistent across both zero-shot and adaptation scenarios, as evidenced by our evaluations on ToxicChat and XSTest. The source code is available at https://github.com/xyq7/GradSafe.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑