OpenAI

How Vulnerable Are AI Agents to Indirect Prompt Injections? Insights from a Large-Scale Public Competition

AI代理对间接提示注入攻击的易受攻击性有多大？来自大规模公开竞赛的见解

Mateusz Dziemian, Maxwell Lin, Xiaohan Fu, Micha Nowak, Nick Winter, Eliot Jones, Andy Zou, Lama Ahmad, Kamalika Chaudhuri, Sahana Chennabasappa, Xander Davies, Lauren Deason, Benjamin L. Edelman, Tanner Emek, Ivan Evtimov, Jim Gust, Maia Hamin, Kat He, Klaudia Krawiecka, Riccardo Patana, Neil Perry, Troy Peterson, Xiangyu Qi, Javier Rando, Zifan Wang, Zihan Wang, Spencer Whitman, Eric Winsor, Arman Zharmagambetov, Matt Fredrikson, Zico Kolter

机构 * Gray Swan AI ； OpenAI ； Meta ； Anthropic ； US CAISI（美国CAISI） ； UK AISI（英国AISI） ； Carnegie Mellon University（卡内基梅隆大学） ； Center for AI Safety（人工智能安全中心） 纠错

AI总结 本文通过大规模公开竞赛评估了AI代理在工具调用、编程和计算机使用中的双重目标，发现所有模型均易受间接提示注入攻击，攻击成功率从0.5%到8.5%不等，并揭示了指令遵循架构的根本性弱点。

Comments 38 pages, 16 figures. Newer version to cover Q1 competition results on latest models in progress. Code at https://github.com/grayswansecurity/ipi_arena_os Partial Dataset at https://huggingface.co/datasets/sureheremarv/ipi_arena_attacks