arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.26754cs.CRcs.AI
收藏

Cordon-MAS:通过信息流控制防御 RAG 的知识投毒

Cordon-MAS: Defending RAG against Knowledge Poisoning via Information-Flow Control

Zhejiang University(浙江大学);Binjiang Institute of Zhejiang University(浙江大学滨江研究院);Shanghai Jiao Tong University(上海交通大学);Zhejiang Lab(浙江实验室);Harbin Institute of Technology(哈尔滨工业大学);Hangzhou Dianzi University(杭州电子科技大学)

Zhe Yu, Wenpeng Xing, Gaolei Li, Shuguang Xiong, Hongzhi Wang, Xuyang Teng, Meng Han

AI总结:

针对检索增强生成(RAG)中的 Confundo 式投毒攻击,提出 Cordon-MAS 框架,通过分离证据提取、跨源审计和答案合成到具有非对称内存权限的智能体中,将攻击成功率相对降低 92.4%,将投毒问题从检测重新定义为信息流控制。

摘要:

检索增强生成(RAG)日益支撑着高风险应用,但仍易受到 Confundo 式投毒攻击,其中对抗性优化的文档操纵生成的输出。现有防御假设检测到中毒证据即可防止危害。我们证明这一假设不正确:模型存在监控-控制差距——它们可以检测到检索证据中的矛盾,但仍会依据中毒声明行动。我们引入 Cordon 原则——任何能够进行最终合成的智能体都不得访问不可信的自然语言证据——并通过 CORDON-MAS 实现该原则,这是一个隔离框架,通过将证据提取、跨源审计和答案合成分离到具有非对称内存权限的智能体中,在架构上强制执行该原则。在五个 BEIR 数据集上,CORDON-MAS 相对于未防御的 RAG 将攻击成功率降低了 92.4%。这将 RAG 投毒问题从检测问题重新定义为信息流控制问题。

英文摘要:

Retrieval-augmented generation (RAG) increasingly underpins high-stakes applications, yet remains vulnerable to Confundo-style poisoning where adversarially optimized documents manipulate generated outputs. Existing defenses assume that detecting poisoned evidence prevents harm. We show this assumption is incorrect: models exhibit a monitoring-control gap -- they can detect contradictions in retrieved evidence yet still act on poisoned claims. We introduce the Cordon Principle -- no agent capable of final synthesis may access untrusted natural-language evidence -- and realize it through CORDON-MAS, a compartmentalized framework that enforces this principle architecturally by separating evidence extraction, cross-source audit, and answer synthesis into agents with asymmetric memory privileges. Across five BEIR datasets, CORDON-MAS reduces attack success rate by 92.4\% relative to undefended RAG. This reframes RAG poisoning from a detection problem to an information-flow control problem.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑