arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.20708cs.AIcs.CR
收藏

ICON:基于推理时校正的代理间接提示注入防御

ICON: Indirect Prompt Injection Defense for Agents based on Inference-Time Correction

College of Computing and Data Science, Nanyang Technological University, Singapore(计算与数据科学学院,南洋理工大学,新加坡);Alibaba(阿里巴巴);School of Computer Science, Peking University, China(计算机科学学院,北京大学,中国)

Che Wang, Fuyao Zhang, Jiaming Zhang, Ziqi Zhang, Yinghui Wang, Longtao Huang, Jianbo Gao, Zhong Chen, Wei Yang Bryan Lim

AI总结:

ICON通过潜在空间轨迹探测和修正器实现对代理间接提示注入攻击的防御,有效提升任务连续性和安全性,同时保持高检测率和任务效用。

摘要:

大型语言模型(LLM)代理容易受到间接提示注入(IPI)攻击,其中检索到的恶意指令会劫持代理的执行。现有防御通常依赖严格的过滤或拒绝机制,但存在关键限制:过度拒绝,提前终止有效的代理工作流程。我们提出了ICON,一种探测到缓解的框架,能够在中性化攻击的同时保持任务连续性。我们的核心见解是IPI攻击在潜在空间中留下明显的过度聚焦签名。我们引入了一个潜在空间轨迹探测器,基于高强度分数来检测攻击。随后,一个缓解修正器执行手术注意力引导,选择性地操纵对抗性查询键依赖关系,同时放大与任务相关的内容以恢复LLM的功能轨迹。在多个backbone上的广泛评估显示,ICON实现了具有竞争力的0.4% ASR,与商业级检测器相匹配,同时产生超过50%的任务效用提升。此外,ICON展示了对分布外(OOD)的鲁棒泛化能力,并有效扩展到多模态代理,建立了安全性和效率之间的优越平衡。

英文摘要:

Large Language Model (LLM) agents are susceptible to Indirect Prompt Injection (IPI) attacks, where malicious instructions in retrieved content hijack the agent's execution. Existing defenses typically rely on strict filtering or refusal mechanisms, which suffer from a critical limitation: over-refusal, prematurely terminating valid agentic workflows. We propose ICON, a probing-to-mitigation framework that neutralizes attacks while preserving task continuity. Our key insight is that IPI attacks leave distinct over-focusing signatures in the latent space. We introduce a Latent Space Trace Prober to detect attacks based on high intensity scores. Subsequently, a Mitigating Rectifier performs surgical attention steering that selectively manipulate adversarial query key dependencies while amplifying task relevant elements to restore the LLM's functional trajectory. Extensive evaluations on multiple backbones show that ICON achieves a competitive 0.4% ASR, matching commercial grade detectors, while yielding a over 50% task utility gain. Furthermore, ICON demonstrates robust Out of Distribution(OOD) generalization and extends effectively to multi-modal agents, establishing a superior balance between security and efficiency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑