arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.11237cs.CL
收藏

用权重量化给大语言模型加水印

Watermarking LLMs with Weight Quantization

Shanghai AI Laboratory(上海人工智能实验室);Fudan University(复旦大学)

Linyang Li, Botian Jiang, Pengyu Wang, Ke Ren, Hang Yan, Xipeng Qiu

AI总结:

本文提出一种在大语言模型量化过程中植入水印的新方法,以保护模型权重,防止未经授权的微调与滥用,并在GPT-Neo和LLaMA上验证了有效性。

摘要:

大语言模型的滥用暴露出高风险,因为大语言模型正以惊人的速度被部署。保护模型权重以避免违反开源大语言模型许可证的恶意使用非常重要。本文提出了一种新颖的水印策略,在大语言模型的量化过程中植入水印,而无需在推理时预定义触发器。当模型以fp32模式使用时,水印生效,而当模型被量化为int8时,水印保持隐藏,这样,用户只能对模型进行推理,而无法进一步对模型进行监督微调。我们成功地将水印植入开源大语言模型权重中,包括GPT-Neo和LLaMA。我们希望我们提出的方法能为在大语言模型应用时代保护模型权重提供一个潜在方向。

英文摘要:

Abuse of large language models reveals high risks as large language models are being deployed at an astonishing speed. It is important to protect the model weights to avoid malicious usage that violates licenses of open-source large language models. This paper proposes a novel watermarking strategy that plants watermarks in the quantization process of large language models without pre-defined triggers during inference. The watermark works when the model is used in the fp32 mode and remains hidden when the model is quantized to int8, in this way, the users can only inference the model without further supervised fine-tuning of the model. We successfully plant the watermark into open-source large language model weights including GPT-Neo and LLaMA. We hope our proposed method can provide a potential direction for protecting model weights in the era of large language model applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑