arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.22242cs.CRcs.AI
收藏

对LLMs对抗提示注入和 jailbreak攻击的分析

Analysis of LLMs Against Prompt Injection and Jailbreak Attacks

Bishop Cotton Boys’ School(伯希特·康普顿男校)

Piyush Jaiswal, Aaditya Pratap, Shreyansh Saraswati, Harsh Kasyap, Somanath Tripathy

AI总结:

本文研究了LLMs对提示注入和jailbreak攻击的脆弱性,通过实验分析不同模型的行为差异,并评估了轻量级防御机制的效果。

摘要:

大型语言模型(LLMs)被广泛应用于现实系统中。鉴于其更广泛的应用性,提示工程已成为资源有限组织采用LLMs用于自身目的的高效工具。同时,LLMs易受基于提示的攻击影响。因此,分析这种风险已成为关键的安全需求。本文通过一个大规模、人工整理的数据集,对多个开源LLMs(包括Phi、Mistral、DeepSeek-R1、Llama 3.2、Qwen和Gemma变体)进行评估,以评估提示注入和jailbreak漏洞。我们观察到模型之间存在显著的行为差异,包括因内部安全机制触发的拒绝响应和完全沉默的无响应。此外,我们评估了几种轻量级、推理时间的防御机制,这些机制在不进行任何重新训练或GPU密集型微调的情况下充当过滤器。尽管这些防御措施可以缓解简单的攻击,但它们始终被长而推理密集的提示所绕过。

英文摘要:

Large Language Models (LLMs) are widely deployed in real-world systems. Given their broader applicability, prompt engineering has become an efficient tool for resource-scarce organizations to adopt LLMs for their own purposes. At the same time, LLMs are vulnerable to prompt-based attacks. Thus, analyzing this risk has become a critical security requirement. This work evaluates prompt-injection and jailbreak vulnerability using a large, manually curated dataset across multiple open-source LLMs, including Phi, Mistral, DeepSeek-R1, Llama 3.2, Qwen, and Gemma variants. We observe significant behavioural variation across models, including refusal responses and complete silent non-responsiveness triggered by internal safety mechanisms. Furthermore, we evaluated several lightweight, inference-time defence mechanisms that operate as filters without any retraining or GPU-intensive fine-tuning. Although these defences mitigate straightforward attacks, they are consistently bypassed by long, reasoning-heavy prompts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑