arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2607.05029cs.CRcs.AI
收藏

你的智能体记忆并非其自身所有:针对大语言模型智能体记忆的伪造推理攻击及防御

Your Agent's Memories Are Not Its Own: Forged Reasoning Attacks on LLM Agent Memory and Defenses

The Pennsylvania State University(宾夕法尼亚州立大学)

Neeraj Karamchandani, Piyush Nagasubramaniam, Sencun Zhu, Dinghao Wu

AI总结:

研究针对大语言模型智能体记忆伪造推理的攻击,提出FARMA攻击方法,通过规避语言伪造推理痕迹并强化,还引入SENTINEL防御管道,实验表明该防御能有效降低攻击成功率。

摘要:

持久内存使大语言模型(LLM)智能体存储多种信息,提升了功能与任务连续性,但也带来新攻击面——智能体自身推理历史。本文介绍伪造放大推理记忆攻击(FARMA),它毒害智能体记忆的推理而非事实知识,还引入SENTINEL检测伪造推理条目,评估显示FARMA攻击成功率高,SENTINEL能有效防御。

英文摘要:

Persistent memory has enabled large language model (LLM) agents to store factual knowledge, prior decisions, reasoning histories, tool usage information, and context. While this has improved the agent's functionality and continuity across tasks, it has also introduced a new attack surface: the agent's own reasoning history. In this paper, we introduce the Forged Amplifying Rationale Memory Attack (FARMA), which poisons an agent's remembered reasoning rather than its factual knowledge. It inserts forged reasoning traces using evasive language that bypasses keyword-based defenses, then amplifies them through self-referential reinforcement that defeats consensus-based defenses. To address FARMA, we introduce SENTINEL, a layered defense pipeline to detect forged reasoning entries. Its central component is the Reasoning Guard that structurally analyzes candidate entries for forgery using five weighted signals. We evaluate FARMA and SENTINEL across multiple agents and different LLM models with 50 trials and show that FARMA achieves an attack success rate of up to 100% under baseline conditions and is capable of defeating defense mechanisms like keyword filter and A-MemGuard. Our evaluation also shows that SENTINEL reduces FARMA's attack success rate to as low as 0% with no false positives observed across 326 benign agent traces. Our work demonstrates the need to protect not only an agent's retrieved content but also the integrity of its reasoning history.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑