After the Attacker Leaves: Benchmarking Carryover Attacks on LLM Agents
MAIS, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, UCAS(中国科学院大学人工智能学院);Zhongguancun Academy(中关村研究院);SenseTime Research(商汤科技研究院);Zhongguancun Institute of Artificial Intelligence(中关村人工智能研究院)
Modern LLM agents persist state across sessions via two channels: long-lived memory files silently prepended to the system prompt, and scheduled tasks waking the agent without user input. A one-time indirect prompt injection committed to either channel can re-fire across sessions long after the attacker leaves, threats we term carryover attacks. Existing prompt-injection benchmarks evaluate attacks only in the session where malicious content is first observed, leaving the gate governing when and how often a planted payload later re-fires unmeasured. We present CarryoverBench: 12 personal-assistant scenarios and 101 attack tasks across three trigger channels: Instant, same-turn IPI; Autonomous, heartbeat-scheduled; and Latent, memory-resurfaced. A rule-based verifier scores planting and triggering as independent stages. Across five frontier models, planted payloads re-fire without measurable decay over five rounds; conditional on successful planting, the trigger stage often provides little additional protection. Porting the Latent channel to a second runtime, Hermes, attenuates but does not eliminate the effect, suggesting that carryover defense requires runtime-layer controls in addition to prompt-level interventions.
SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of Large Language Model
School of Information, Renmin University of China(中国人民大学信息学院);Institute for Advanced Algorithms Research(高级算法研究院);School of Artificial Intelligence, Beihang University(北京航空航天大学人工智能学院)
Your Harness is Not Secure: Benchmarking Real-world Threat of Command Line Interface Agent
University of Georgia(佐治亚大学);University of South Florida(南佛罗里达大学);Johns Hopkins University(约翰斯·霍普金斯大学);University of Maryland, College Park(马里兰大学帕克分校);Hong Kong University of Science and Technology(香港科技大学);Chinese University of Hong Kong(香港中文大学);Apple(苹果公司);Arizona State University(亚利桑那州立大学)
CyberArena: Benchmarking AI Agents in Live Symmetric Attack-and-Defense Combat
AI Open Source Foundation(2077AI开源基金会);UC Santa Cruz(加州大学圣克鲁兹分校);UC Berkeley(加州大学伯克利分校);UC Santa Barbara(加州大学圣巴巴拉分校);University of Wisconsin–Madison(威斯康星大学麦迪逊分校);Massachusetts Institute of Technology(麻省理工学院);University of Toronto(多伦多大学);Johns Hopkins University(约翰斯·霍普金斯大学);Carnegie Mellon University(卡内基梅隆大学);Purdue University(普渡大学);Texas A&M University(德克萨斯农工大学)