arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.05249cs.CRcs.AI
收藏

RAG-WM:一种面向大语言模型检索增强生成的高效黑盒水印方法

RAG-WM: An Efficient Black-Box Watermarking Approach for Retrieval-Augmented Generation of Large Language Models

Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Science and Technology, Shandong University(山东大学网络空间安全学院);Indiana University Bloomington(印第安纳大学伯明顿分校);Department of Computer Science, Metropolitan College, Boston University(波士顿大学大都会学院计算机科学系);Institute for Clarity in Documentation(文档清晰研究所);Inria Paris-Rocquencourt(法国国家信息与自动化研究所巴黎-罗康分所);Rajiv Gandhi University(拉吉夫·甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕尔默研究实验室)

Peizhuo Lv, Mengjie Sun, Hao Wang, Xiaofeng Wang, Shengzhi Zhang, Yuxuan Chen, Kai Chen, Limin Sun

AI总结:

针对RAG知识库易被窃取且现有白盒数据库水印、文本水印难以适用的问题,本文提出黑盒知识水印框架RAG-WM,通过多LLM交互注入实体-关系水印并验证侵权,实验证明其有效且鲁棒、隐蔽。

摘要:

近年来,检索增强生成(RAG)取得了巨大成功,并被广泛用于增强大语言模型(LLM)在特定领域、知识密集型和隐私敏感任务中的能力。然而,攻击者可能窃取这些有价值的RAG并将其部署或商业化,因此检测知识产权(IP)侵权变得至关重要。现有的所有权保护方案(如水印)大多面向关系数据库和文本设计。它们无法直接应用于RAG,因为关系数据库水印需要白盒访问才能检测知识产权侵权,这对RAG中的知识库而言并不现实;同时,攻击者部署的LLM进行的后处理通常会破坏文本水印信息。为解决这些问题,我们提出一种名为RAG-WM的新型黑盒“知识水印”方法,用于检测RAG的知识产权侵权。RAG-WM采用多LLM交互框架,由Watermark Generator、Shadow LLM & RAG和Watermark Discriminator组成,基于水印实体-关系元组创建水印文本,并将其注入目标RAG。我们在四个基准LLM上,针对三个特定领域任务和两个隐私敏感任务评估RAG-WM。实验结果表明,RAG-WM能够在各种部署的LLM中有效检测被窃取的RAG。此外,RAG-WM对释义、无关内容删除、知识插入和知识扩展攻击具有鲁棒性。最后,RAG-WM还能够规避水印检测方法,凸显了其在检测RAG系统知识产权侵权方面的良好应用前景。

英文摘要:

In recent years, tremendous success has been witnessed in Retrieval-Augmented Generation (RAG), widely used to enhance Large Language Models (LLMs) in domain-specific, knowledge-intensive, and privacy-sensitive tasks. However, attackers may steal those valuable RAGs and deploy or commercialize them, making it essential to detect Intellectual Property (IP) infringement. Most existing ownership protection solutions, such as watermarks, are designed for relational databases and texts. They cannot be directly applied to RAGs because relational database watermarks require white-box access to detect IP infringement, which is unrealistic for the knowledge base in RAGs. Meanwhile, post-processing by the adversary's deployed LLMs typically destructs text watermark information. To address those problems, we propose a novel black-box "knowledge watermark" approach, named RAG-WM, to detect IP infringement of RAGs. RAG-WM uses a multi-LLM interaction framework, comprising a Watermark Generator, Shadow LLM & RAG, and Watermark Discriminator, to create watermark texts based on watermark entity-relationship tuples and inject them into the target RAG. We evaluate RAG-WM across three domain-specific and two privacy-sensitive tasks on four benchmark LLMs. Experimental results show that RAG-WM effectively detects the stolen RAGs in various deployed LLMs. Furthermore, RAG-WM is robust against paraphrasing, unrelated content removal, knowledge insertion, and knowledge expansion attacks. Lastly, RAG-WM can also evade watermark detection approaches, highlighting its promising application in detecting IP infringement of RAG systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑