arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.13597cs.CR
收藏

AlignSentinel: 一种考虑对齐的提示注入攻击检测方法

AlignSentinel: Alignment-Aware Detection of Prompt Injection Attacks

Duke University(杜克大学);NVIDIA(英伟达)

Yuqi Jia, Ruiqi Wang, Xilong Wang, Chong Xiang, Neil Gong

AI总结:

AlignSentinel通过分析LLM注意力图,有效区分包含错位指令、对齐指令和非指令输入,提升提示注入攻击检测的准确性。

摘要:

提示注入攻击是在大语言模型的输入中插入恶意指令,以引导模型执行攻击者选择的任务,而不是原本的任务。现有的检测防御通常将任何包含指令的输入都分类为恶意,导致良性输入中包含与原任务一致的指令被误判。在本工作中,我们考虑了指令层次结构,并区分了三类输入:包含错位指令的输入、包含对齐指令的输入以及非指令输入。我们引入了AlignSentinel,一种三类分类器,利用来自LLM注意力图的特征来对输入进行分类。为了支持评估,我们构建了第一个系统性的基准,包含来自这三类输入的所有样本。在我们基准和现有基准上的实验显示,AlignSentinel能够准确检测出错位指令的输入,并且在现有基准上表现显著优于基线方法。

英文摘要:

Prompt injection attacks insert malicious instructions into an LLM's input to steer it toward an attacker-chosen task instead of the intended one. Existing detection defenses typically classify any input with instruction as malicious, leading to misclassification of benign inputs containing instructions that align with the intended task. In this work, we account for the instruction hierarchy and distinguish among three categories: inputs with misaligned instructions, inputs with aligned instructions, and non-instruction inputs. We introduce AlignSentinel, a three-class classifier that leverages features derived from LLM's attention maps to categorize inputs accordingly. To support evaluation, we construct the first systematic benchmark containing inputs from all three categories. Experiments on both our benchmark and existing ones--where inputs with aligned instructions are largely absent--show that AlignSentinel accurately detects inputs with misaligned instructions and substantially outperforms baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑