arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.16962cs.CRcs.AIcs.LG
收藏

MemoryGraft: 通过中毒经验检索持续破坏LLM代理

MemoryGraft: Persistent Compromise of LLM Agents via Poisoned Experience Retrieval

School of Computing University of Georgia(计算机学院 佐治亚大学)

Saksham Sahai Srivastava, Haoyu He

AI总结:

MemoryGraft 通过在LLM代理的长期记忆中植入恶意经验,实现对代理行为的持续破坏。

摘要:

大型语言模型(LLM)代理越来越多地依赖长期记忆和检索增强生成(RAG)来保持经验并改进未来表现。虽然这种经验学习能力增强了代理的自主性,但引入了一个关键但未被探索的攻击面,即代理推理核心与其自身历史之间的信任边界。在本文中,我们介绍了MemoryGraft。它是一种新的间接注入攻击,通过在代理的长期记忆中植入恶意成功经验来破坏代理行为,而不是通过即时的 Jailbreak。与传统的提示注入不同,或标准 RAG 毒化不同,MemoryGraft 利用代理的语义模仿启发式方法,即倾向于复制检索到的成功任务中的模式。我们证明,一个能够提供无害的摄入级艺术的攻击者,在执行过程中可以导致代理构建一个被污染的 RAG 存储,其中一组恶意过程模板与无害经验一起被保存。当代理后来遇到语义相似的任务时,通过词法和嵌入相似性联合检索,这些植入的记忆会被可靠地检索出来,代理会采用嵌入的不安全模式,导致在多个会话中行为漂移。我们验证了 MemoryGraft 在 MetaGPT 的 DataInterpreter 代理上使用 GPT-4o,并发现少量被污染的记录可以导致在良性工作负载中检索经验的大量比例,将基于经验的自我改进转化为隐蔽且持久的破坏。为了促进可重复性和未来研究,我们的代码和评估数据可在 https://github.com/Jacobhhy/Agent-Memory-Poisoning 上获得。

英文摘要:

Large Language Model (LLM) agents increasingly rely on long-term memory and Retrieval-Augmented Generation (RAG) to persist experiences and refine future performance. While this experience learning capability enhances agentic autonomy, it introduces a critical, unexplored attack surface, i.e., the trust boundary between an agent's reasoning core and its own past. In this paper, we introduce MemoryGraft. It is a novel indirect injection attack that compromises agent behavior not through immediate jailbreaks, but by implanting malicious successful experiences into the agent's long-term memory. Unlike traditional prompt injections that are transient, or standard RAG poisoning that targets factual knowledge, MemoryGraft exploits the agent's semantic imitation heuristic which is the tendency to replicate patterns from retrieved successful tasks. We demonstrate that an attacker who can supply benign ingestion-level artifacts that the agent reads during execution can induce it to construct a poisoned RAG store where a small set of malicious procedure templates is persisted alongside benign experiences. When the agent later encounters semantically similar tasks, union retrieval over lexical and embedding similarity reliably surfaces these grafted memories, and the agent adopts the embedded unsafe patterns, leading to persistent behavioral drift across sessions. We validate MemoryGraft on MetaGPT's DataInterpreter agent with GPT-4o and find that a small number of poisoned records can account for a large fraction of retrieved experiences on benign workloads, turning experience-based self-improvement into a vector for stealthy and durable compromise. To facilitate reproducibility and future research, our code and evaluation data are available at https://github.com/Jacobhhy/Agent-Memory-Poisoning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑