发表机构
Southwestern University of Finance and Economics; Nanjing University of Science and Technology（西南财经大学; 南京理工大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出Vis-Poison攻击，通过自动化多智能体方法构建视觉合理的被投毒图像，在黑盒设置下对多模态RAG系统实现40.16%-65.40%的攻击成功率，且对仅依赖参数知识的MLLM平均成功率超60%。
AI 中文摘要
多模态检索增强生成（RAG）系统日益依赖图像作为外部知识来源，而投毒的视觉证据会严重损害多模态大语言模型（MLLM）的生成结果。与以往依赖修改文本元数据的攻击不同，本文提出Vis-Poison，一种新型视觉知识投毒攻击，其中被投毒图像本身是攻击者控制的有效载荷，无需操作标题、摘要、元数据或其他相关文本。具体而言，该攻击通过一种自动化多智能体方法实现，该方法可构建视觉上合理的被投毒图像。为评估其影响，本文在两种代表性多模态RAG管道、四种嵌入模型和六种生成模型上对Vis-Poison进行评估。实验表明，在黑盒设置下，针对3万条条目的多模态知识库，Vis-Poison实现了40.16%至65.40%的端到端攻击成功率。此外，Vis-Poison对仅能通过参数知识正确回答的各种MLLM仍然有效，平均成功率超过60%。代码和数据可在此https URL获取。
英文摘要
While multimodal retrieval-augmented generation (RAG) systems increasingly rely on images as external knowledge sources, the introduction of poisoned visual evidence can severely compromise multimodal large language model (MLLM) generation. Unlike prior attacks that rely on altering textual metadata, we introduce Vis-Poison, a novel visual knowledge poisoning attack where the poisoned image itself is the attacker-controlled payload, without manipulating captions, summaries, metadata, or other associated text. Specifically, this attack is instantiated through an automated multi-agent method that constructs visually plausible poisoned images. To assess its impact, we evaluate Vis-Poison across two representative multimodal RAG pipelines, four embedding models, and six generation models. Empirically, Vis-Poison achieves an end-to-end attack success rate of 40.16% to 65.40% against 30k-entry multimodal knowledge bases in \emph{black-box} settings. Moreover, Vis-Poison remains effective against various MLLMs that can answer correctly from parametric knowledge alone, with an average success rate above 60%. Code and data are available at https://github.com/SWUFE-DB-Group/Vis-Poison.
CommentsFindings of EMNLP, 2026
