arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.16029cs.CRcs.AI
收藏

FDLLM:一种专为黑盒LLM指纹识别设计的检测器

FDLLM: A Dedicated Detector for Black-Box LLMs Fingerprinting

College of Cyberspace Security, Hainan University(海南大学网络安全学院);National Computer Network Intrusion Protection Center, University of Chinese Academy of Sciences(中国科学院大学国家计算机网络入侵防护中心);Graduate School of Natural Science and Technology, Kanazawa University(北九州市立大学自然科学技术研究生院);School of Cyber Engineering, Xidian University(西安电子科技大学 cyber 工程学院);School of Informatics, Computing, and Cyber Systems, Northern Arizona University(北亚利桑那大学信息学、计算与网络系统学院);College of IST, Pennsylvania State University(宾夕法尼亚州立大学信息科技学院)

Zhiyuan Fu, Junfan Chen, Lan Zhang, Ting Yang, Jun Niu, Hongyu Sun, Ruidong Li, Peng Liu, Jice Wang, Fannv He, Qiuling Yue, Yuqing Zhang

AI总结:

FDLLM通过LoRA技术实现高效LLM指纹识别,具备高准确率和鲁棒性,有效提升模型识别与安全防护能力。

摘要:

大型语言模型(LLMs)正在迅速改变数字内容创作的格局。然而,许多LLM盛行的黑盒应用程序编程接口(API)访问引入了显著的责任、治理和安全挑战。LLM指纹识别旨在通过分析生成文本的统计和风格特征来识别源模型,提供了一种潜在的解决方案。当前在这一领域的进展受到缺乏专门数据集和需要高效、实用且对对抗操纵具有鲁棒性的方法的阻碍。为了解决这些挑战,我们介绍了FD-Dataset,一个包含90,000个文本样本的双语指纹识别基准,涵盖20个著名的专有和开源LLM。此外,我们提出了FDLLM,一种新的指纹识别方法,利用参数高效的低秩适应(LoRA)来微调基础模型。这种方法使LoRA能够提取深度、持久的特征,以表征每个源LLM。通过我们的分析,我们发现LoRA适应促进了相同LLM在表示空间中的输出聚合,同时增强了不同LLM之间的分离。这种机制解释了为什么LoRA在LLM指纹识别中特别有效。在FD-Dataset上的广泛实证评估证明了FDLLM的优越性,其宏F1得分比最强基线高22.1%。FDLLM还表现出对新发布模型的良好泛化能力,在未见过的模型上实现了95%的平均准确率。值得注意的是,FDLLM在各种对抗性攻击下保持稳定,包括打磨、翻译和同义词替换。实验结果表明,FDLLM将平均攻击成功率从49.2%(LM-D)降低到23.9%。

英文摘要:

Large Language Models (LLMs) are rapidly transforming the landscape of digital content creation. However, the prevalent black-box Application Programming Interface (API) access to many LLMs introduces significant challenges in accountability, governance, and security. LLM fingerprinting, which aims to identify the source model by analyzing statistical and stylistic features of generated text, offers a potential solution. Current progress in this area is hindered by a lack of dedicated datasets and the need for efficient, practical methods that are robust against adversarial manipulations. To address these challenges, we introduce FD-Dataset, a comprehensive bilingual fingerprinting benchmark comprising 90,000 text samples from 20 famous proprietary and open-source LLMs. Furthermore, we present FDLLM, a novel fingerprinting method that leverages parameter-efficient Low-Rank Adaptation (LoRA) to fine-tune a foundation model. This approach enables LoRA to extract deep, persistent features that characterize each source LLM. Through our analysis, we find that LoRA adaptation promotes the aggregation of outputs from the same LLM in representation space while enhancing the separation between different LLMs. This mechanism explains why LoRA proves particularly effective for LLM fingerprinting. Extensive empirical evaluations on FD-Dataset demonstrate FDLLM's superiority, achieving a Macro F1 score 22.1% higher than the strongest baseline. FDLLM also exhibits strong generalization to newly released models, achieving an average accuracy of 95% on unseen models. Notably, FDLLM remains consistently robust under various adversarial attacks, including polishing, translation, and synonym substitution. Experimental results show that FDLLM reduces the average attack success rate from 49.2% (LM-D) to 23.9%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑