arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.07760cs.CRcs.LG
收藏

大语言模型的可扩展指纹技术

Scalable Fingerprinting of Large Language Models

University of Washington(华盛顿大学);Sentient

Anshul Nasery, Jonathan Hayase, Creston Brooks, Peiyao Sheng, Himanshu Tyagi, Pramod Viswanath, Sewoong Oh

AI总结:

该研究针对大语言模型指纹的可扩展性需求,提出Perinucleus采样方法,可在Llama-3.1-8B中嵌入比现有方案多两个数量级的24576个持久无害指纹,且不降低模型效用,还能缓解相关安全风险。

摘要:

模型指纹已成为模型所有者在仅拥有API访问权限的情况下识别其共享模型的有力工具。然而,为了降低误发现率、防范指纹泄露,以及抵御试图绕过检测的模型用户联盟,我们认为**可扩展性**至关重要,即提升可嵌入模型的指纹数量。因此,我们将可扩展性列为指纹方案的一项核心要求。我们在远大于以往研究的规模下开展指纹设计实验,并提出一种名为Perinucleus采样的新方法,用于生成可扩展、持久且无害的指纹。我们证明该方案可向Llama-3.1-8B模型中添加24576个指纹——比现有方案高出两个数量级——且不会降低模型的效用。我们插入的指纹即使在标准训练后数据上进行监督微调后依然存在。我们进一步探讨了指纹技术的安全风险,并从理论和实证层面展示了我们这类可扩展指纹方案如何缓解这些风险。我们的代码可在https://github.com/SewoongLab/scalable-fingerprinting-of-llms获取。

英文摘要:

Model fingerprinting has emerged as a powerful tool for model owners to identify their shared model given API access. However, to lower false discovery rate, fight fingerprint leakage, and defend against coalitions of model users attempting to bypass detection, we argue that {\em scalability} is critical, i.e., scaling up the number of fingerprints one can embed into a model. Hence, we pose scalability as a crucial requirement for fingerprinting schemes. We experiment with fingerprint design at a scale significantly larger than previously considered, and introduce a new method, dubbed Perinucleus sampling, to generate scalable, persistent, and harmless fingerprints. We demonstrate that this scheme can add 24,576 fingerprints to a Llama-3.1-8B model -- two orders of magnitude more than existing schemes -- without degrading the model's utility. Our inserted fingerprints persist even after supervised fine-tuning on standard post-training data. We further address security risks for fingerprinting, and theoretically and empirically show how a scalable fingerprinting scheme like ours can mitigate these risks. Our code is available at https://github.com/SewoongLab/scalable-fingerprinting-of-llms

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑