arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.10481cs.CRcs.AIcs.MA
收藏

保护上下文和提示:非确定性AI的确定性安全

Protecting Context and Prompts: Deterministic Security for Non-Deterministic AI

MACAW Security, Inc.(MACAW安全公司)

Mohan Rajagopalan, Vinay Rao

AI总结:

本研究提出认证提示和上下文技术,通过加密验证和策略代数实现LLM的预防性安全,有效防止提示注入和上下文操纵攻击。

摘要:

大型语言模型(LLM)应用易受提示注入和上下文操纵攻击的侵害,这些攻击传统安全模型无法阻止。我们引入两个新的原始技术——认证提示和认证上下文——它们为LLM工作流程提供加密可验证的来源。认证提示使自我包含的来源验证成为可能,而认证上下文使用篡改证据哈希链来确保动态输入的完整性。基于这些原始技术,我们正式化了一种策略代数,通过四个定理提供协议层面的拜占庭容错性——即使有敌对代理也无法违反组织政策。五种互补的防御措施——从轻量级资源控制到基于LLM的语义验证——提供了分层的预防性安全,具有形式保证。对代表性攻击的评估覆盖了6种详尽的类别,实现了100%的检测率,零误报,并具有名义上的开销。我们展示了首次结合加密强制的提示来源、篡改证据上下文和可证明的策略推理的方法——将LLM安全从反应性检测转向预防性保证。

英文摘要:

Large Language Model (LLM) applications are vulnerable to prompt injection and context manipulation attacks that traditional security models cannot prevent. We introduce two novel primitives--authenticated prompts and authenticated context--that provide cryptographically verifiable provenance across LLM workflows. Authenticated prompts enable self-contained lineage verification, while authenticated context uses tamper-evident hash chains to ensure integrity of dynamic inputs. Building on these primitives, we formalize a policy algebra with four proven theorems providing protocol-level Byzantine resistance--even adversarial agents cannot violate organizational policies. Five complementary defenses--from lightweight resource controls to LLM-based semantic validation--deliver layered, preventative security with formal guarantees. Evaluation against representative attacks spanning 6 exhaustive categories achieves 100% detection with zero false positives and nominal overhead. We demonstrate the first approach combining cryptographically enforced prompt lineage, tamper-evident context, and provable policy reasoning--shifting LLM security from reactive detection to preventative guarantees.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑