We introduce LLMmap, a first-generation fingerprinting technique targeted at LLM-integrated applications. LLMmap employs an active fingerprinting approach, sending carefully crafted queries to the application and analyzing the responses to identify the specific LLM version in use. Our query selection is informed by domain expertise on how LLMs generate uniquely identifiable responses to thematically varied prompts. With as few as 8 interactions, LLMmap can accurately identify 42 different LLM versions with over 95% accuracy. More importantly, LLMmap is designed to be robust across different application layers, allowing it to identify LLM versions--whether open-source or proprietary--from various vendors, operating under various unknown system prompts, stochastic sampling hyperparameters, and even complex generation frameworks such as RAG or Chain-of-Thought. We discuss potential mitigations and demonstrate that, against resourceful adversaries, effective countermeasures may be challenging or even unrealizable.
DNF: Dual-Layer Nested Fingerprinting for Large Language Model Intellectual Property Protection
Zhejiang University(浙江大学);Binjiang Institute of Zhejiang University(浙江大学滨江学院);Hangzhou Dianzi University(杭州电子科技大学);Hefei University of Technology(合肥工业大学)
SoK: Large Language Model Copyright Auditing via Fingerprinting
The State Key Laboratory of Blockchain and Data Security(区块链与数据安全国家重点实验室);Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security(杭州高新技术区(滨江)区块链与数据安全研究院);Nanyang Technological University(南洋理工大学);City University of Hong Kong(香港城市大学);Sun Yat-Sen University(中山大学)
FDLLM: A Dedicated Detector for Black-Box LLMs Fingerprinting
College of Cyberspace Security, Hainan University(海南大学网络安全学院);National Computer Network Intrusion Protection Center, University of Chinese Academy of Sciences(中国科学院大学国家计算机网络入侵防护中心);Graduate School of Natural Science and Technology, Kanazawa University(北九州市立大学自然科学技术研究生院);School of Cyber Engineering, Xidian University(西安电子科技大学 cyber 工程学院);School of Informatics, Computing, and Cyber Systems, Northern Arizona University(北亚利桑那大学信息学、计算与网络系统学院);College of IST, Pennsylvania State University(宾夕法尼亚州立大学信息科技学院)
LLMs Have Rhythm: Fingerprinting Large Language Models Using Inter-Token Times and Network Traffic Analysis
College of Science and Engineering (CSE), Hamad Bin Khalifa University (HBKU)(哈马德·本·卡伊夫大学科学与工程学院);Networked Systems Security Group, KTH Royal Institute of Technology -- Stockholm, Sweden(瑞典皇家理工学院网络系统安全组)
Prompt2Fingerprint: Plug-and-Play LLM Fingerprinting via Text-to-Weight Generation
Shenzhen International Graduate School, Tsinghua University, Shenzhen, China(清华大学深圳国际研究生院,中国深圳);South China University of Technology, Guangzhou, China(华南理工大学,中国广州);Harbin Institute of Technology, Shenzhen, Shenzhen, China(哈尔滨工业大学深圳校区,中国深圳)
A Large-Scale Benchmark and Risk Assessment of Traffic Analysis Attacks on Cloud LLM Services
Chalmers University of Technology(查尔姆斯理工大学);University of Texas at El Paso(德克萨斯大学埃尔帕索分校);Old Dominion University(奥多明尼昂大学);George Mason University(乔治梅森大学)
From Bits to Beliefs: Recoverable Semantic Fingerprints for Black-Box Verification of Large Language Models
Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));South China University of Technology(华南理工大学);Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院)
Have You Merged My Model? On The Robustness of Large Language Model IP Protection Methods Against Model Merging
Tsinghua University(清华大学);Xidian University(西安电子科技大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Zhongguancun Laboratory(中关村实验室);National Financial Cryptography Research Center(国家金融密码研究中心)
KinGuard: Hierarchical Kinship-Aware Fingerprinting to Defend Against Large Language Model Stealing
Zhejiang University(浙江大学);The University of Melbourne(墨尔本大学);National University of Singapore(新加坡国立大学);People’s Public Security University of China(中国人民公安大学);Shanghai Jiao Tong University(上海交通大学)
TRAP: Targeted Random Adversarial Prompt Honeypot for Black-Box Identification
Parameter Lab(参数实验室);IT University of Copenhagen(哥本哈根信息技术大学);Pioneer Centre for Artificial Intelligence(人工智能先锋中心);NAVER AI Lab(NAVER人工智能实验室);University of Tübingen(蒂宾根大学)
The Faithfulness Gap: Certifying Semantic Equivalence Between Natural-Language and Formal Mathematical Statements
Department of Computer Science, Informatics Institute, Istanbul Technical University, İstanbul, Türkiye(信息学院计算机科学系,伊斯坦布尔技术大学,伊斯坦布尔,土耳其);Department of Computer Science(计算机科学系);Engineering, Jashore University of Science(工程系,贾沙尔大学科学学院)
Fingerprinting Text-to-Image Diffusion Models via Collapsed Generation
Fudan University(复旦大学);East China University of Science and Technology(华东理工大学);Alibaba Group(阿里巴巴集团);Shanghai Pudong Research Institute of Cryptology(上海浦东密码学研究所)
AttnDiff: Attention-based Differential Fingerprinting for Large Language Models
Zhejiang University of Technology(浙江工业大学);Zhejiang University(浙江大学);Binjiang Institute of Zhejiang University(浙江大学滨江研究院);Shanghai Jiao Tong University(上海交通大学);University of Science and Technology of China(中国科学技术大学)