零样本嵌入漂移检测：一种轻量级防御对抗提示注入的LLM方法
Zero-Shot Embedding Drift Detection: A Lightweight Defense Against Prompt Injections in LLMs
AI总结：
本文提出ZEDD，通过量化嵌入空间中良性与可疑输入之间的语义变化，实现对直接和间接提示注入的检测。该方法无需模型内部访问或先验知识，具有低工程开销，能高效部署于多种LLM架构，准确率达93%以上。
摘要：
提示注入攻击已成为LLM应用中的日益严重漏洞，其中对抗性提示利用电子邮件或用户生成内容等间接输入渠道绕过对齐保护措施，导致有害或意外输出。尽管对齐技术有所进步，但最先进的LLM仍广泛易受对抗性提示攻击，凸显了需要稳健、高效且可推广的检测机制的紧迫性。本文提出零样本嵌入漂移检测（ZEDD），一种轻量级、低工程开销的框架，通过量化嵌入空间中良性与可疑输入之间的语义变化，识别直接和间接提示注入尝试。ZEDD无需访问模型内部、先验攻击类型知识或任务特定重训练，可高效地在多种LLM架构上进行零样本部署。我们的方法使用对抗性清洁提示对，并通过余弦相似度测量嵌入漂移，以捕捉现实世界注入攻击中的细微对抗性操纵。为确保评估的鲁棒性，我们编纂并重新标注了涵盖五个注入类别的综合LLMail-Inject数据集。广泛实验表明，嵌入漂移是一种稳健且可转移的信号，优于传统方法在检测准确性和操作效率方面。在Llama 3、Qwen 2和Mistral等模型架构上，分类准确率超过93%，误报率低于3%，我们的方法提供了一种轻量级、可扩展的防御层，可整合到现有LLM流程中，填补了保护LLM系统以抵御适应性对抗威胁的关键空白。
英文摘要：
Prompt injection attacks have become an increasing vulnerability for LLM applications, where adversarial prompts exploit indirect input channels such as emails or user-generated content to circumvent alignment safeguards and induce harmful or unintended outputs. Despite advances in alignment, even state-of-the-art LLMs remain broadly vulnerable to adversarial prompts, underscoring the urgent need for robust, productive, and generalizable detection mechanisms beyond inefficient, model-specific patches. In this work, we propose Zero-Shot Embedding Drift Detection (ZEDD), a lightweight, low-engineering-overhead framework that identifies both direct and indirect prompt injection attempts by quantifying semantic shifts in embedding space between benign and suspect inputs. ZEDD operates without requiring access to model internals, prior knowledge of attack types, or task-specific retraining, enabling efficient zero-shot deployment across diverse LLM architectures. Our method uses adversarial-clean prompt pairs and measures embedding drift via cosine similarity to capture subtle adversarial manipulations inherent to real-world injection attacks. To ensure robust evaluation, we assemble and re-annotate the comprehensive LLMail-Inject dataset spanning five injection categories derived from publicly available sources. Extensive experiments demonstrate that embedding drift is a robust and transferable signal, outperforming traditional methods in detection accuracy and operational efficiency. With greater than 93% accuracy in classifying prompt injections across model architectures like Llama 3, Qwen 2, and Mistral and a false positive rate of <3%, our approach offers a lightweight, scalable defense layer that integrates into existing LLM pipelines, addressing a critical gap in securing LLM-powered systems to withstand adaptive adversarial threats.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
