ICON: Indirect Prompt Injection Defense for Agents based on Inference-Time Correction
College of Computing and Data Science, Nanyang Technological University, Singapore(计算与数据科学学院,南洋理工大学,新加坡);Alibaba(阿里巴巴);School of Computer Science, Peking University, China(计算机科学学院,北京大学,中国)
Che Wang, Fuyao Zhang, Jiaming Zhang, Ziqi Zhang, Yinghui Wang, Longtao Huang, Jianbo Gao, Zhong Chen, Wei Yang Bryan Lim
Large Language Model (LLM) agents are susceptible to Indirect Prompt Injection (IPI) attacks, where malicious instructions in retrieved content hijack the agent's execution. Existing defenses typically rely on strict filtering or refusal mechanisms, which suffer from a critical limitation: over-refusal, prematurely terminating valid agentic workflows. We propose ICON, a probing-to-mitigation framework that neutralizes attacks while preserving task continuity. Our key insight is that IPI attacks leave distinct over-focusing signatures in the latent space. We introduce a Latent Space Trace Prober to detect attacks based on high intensity scores. Subsequently, a Mitigating Rectifier performs surgical attention steering that selectively manipulate adversarial query key dependencies while amplifying task relevant elements to restore the LLM's functional trajectory. Extensive evaluations on multiple backbones show that ICON achieves a competitive 0.4% ASR, matching commercial grade detectors, while yielding a over 50% task utility gain. Furthermore, ICON demonstrates robust Out of Distribution(OOD) generalization and extends effectively to multi-modal agents, establishing a superior balance between security and efficiency.
Taxonomy, Evaluation and Exploitation of IPI-Centric LLM Agent Defense Frameworks
The Hong Kong University of Science and Technology(香港科技大学);Zhejiang University of Technology(浙江工业大学);Lingnan University(岭南大学);School of Cyber Science and Engineering, Southeast University(东南大学计算机科学与工程学院);ZTE Corporation(中兴通讯有限公司)
AgentSentry: Mitigating Indirect Prompt Injection in LLM Agents via Temporal Causal Diagnostics and Context Purification
Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University(航空信息安全与可信计算重点实验室、教育部、网络安全科学与工程学院、武汉大学);Department of Computer Science and Engineering, University at Buffalo, SUNY(计算机科学与工程系、布法罗大学、纽约州立大学);Department of Computer Science, College of Information Science and Technology, Jinan University(计算机科学系、信息科学与技术学院、济南大学);College of Cyber Security, Jinan University(网络安全学院、济南大学)
ActGuard: Pre-execution Action Auditing against Indirect Prompt Injection in LLM Agents
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);School of Cyber Science and Technology, Beihang University(北京航空航天大学网络空间安全学院);University of Wisconsin–Madison(威斯康星大学麦迪逊分校)
Multi-Stage Prompt Inference Attacks on Enterprise LLM Systems
Ukrainian State University of Science and Technologies, ESI "Prydniprovska State Academy of Civil Engineering and Architecture(乌克兰科学与技术国家大学,埃斯I普里皮亚特斯卡州立建筑与工程学院);Google DeepMind(谷歌DeepMind)
RAISED: Self-Distillation for Robustness to Prompt Injection in LLM Agents
École polytechnique(巴黎综合理工学院);Institut Polytechnique de Paris(巴黎理工学院);CNRS(法国国家科学研究中心);Google DeepMind(谷歌DeepMind);Agence Ministérielle pour l’IA de Défense(法国国防部人工智能局);IRT SystemX(法国系统X技术研究所)
APEX: Active Protection at Execution Boundaries for LLM Agents
Tsinghua University(清华大学);Imperial College London(帝国理工学院);Nanjing University(南京大学);The Chinese University of Hong Kong(香港中文大学);University College London(伦敦大学学院);A*STAR(新加坡科技研究局);University of British Columbia(不列颠哥伦比亚大学);Shenzhen University(深圳大学);The University of Hong Kong(香港大学)
AgentRAE: Remote Action Execution through Notification-based Visual Backdoors against Screenshots-based Mobile GUI Agents
Nanjing University of Science and Technology(南京理工大学);Macquarie University(麦考瑞大学);Western Sydney University(西澳大学);University of Hong Kong(香港大学);CSIRO’s Data61(澳大利亚CSIRO Data61)
Less Diverse, Less Safe: The Indirect But Pervasive Risk of Test-Time Scaling in Large Language Models
Bellini College of AI, Cybersecurity, and Computing(人工智能、网络安全与计算学院);University of South Florida, Tampa, Florida, USA(佛罗里达州塔帕斯大学);University of California, Davis, California, USA(加州大学戴维斯分校)
AdapTools: Adaptive Tool-based Indirect Prompt Injection Attacks on Agentic LLMs
College of Computing(计算学院);Data Science, Nanyang Technological University, Singapore(数据科学,南洋理工大学,新加坡);School of Computer Science, Peking University, China(计算机科学学院,北京大学,中国)
Defense without Forgetting: Continual Adversarial Defense with Anisotropic & Isotropic Pseudo Replay
Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies(广东省新型安全智能技术重点实验室)