arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.15847cs.CRcs.AI
收藏

LLMmap:面向大型语言模型的指纹识别技术

LLMmap: Fingerprinting For Large Language Models

RSAC Labs(RSAC实验室);George Mason University(乔治梅森大学)

Dario Pasquini, Evgenios M. Kornaropoulos, Giuseppe Ateniese

AI总结:

该研究提出LLMmap这一针对集成LLM应用的指纹识别技术,通过主动发送精心设计的查询并分析响应,以最少8次交互、95%以上准确率识别42种LLM版本,且对不同应用层鲁棒。

摘要:

我们提出LLMmap,这是针对集成大型语言模型(LLM)的应用程序的第一代指纹识别技术。LLMmap采用主动指纹识别方法,向应用程序发送精心设计的查询并分析响应,以识别所用的特定LLM版本。我们的查询选择基于领域专业知识,即LLM如何对主题多样的提示生成唯一可识别的响应。仅需8次交互,LLMmap就能以超过95%的准确率准确识别42种不同的LLM版本。更重要的是,LLMmap设计为在不同应用层具有鲁棒性,使其能够从各种供应商的应用程序中识别LLM版本——无论是开源还是专有版本,无论其运行在未知的系统提示、随机采样超参数下,甚至在RAG或Chain-of-Thought等复杂生成框架中。我们讨论了潜在的缓解措施,并证明面对资源丰富的攻击者,有效的对策可能具有挑战性甚至无法实现。

英文摘要:

We introduce LLMmap, a first-generation fingerprinting technique targeted at LLM-integrated applications. LLMmap employs an active fingerprinting approach, sending carefully crafted queries to the application and analyzing the responses to identify the specific LLM version in use. Our query selection is informed by domain expertise on how LLMs generate uniquely identifiable responses to thematically varied prompts. With as few as 8 interactions, LLMmap can accurately identify 42 different LLM versions with over 95% accuracy. More importantly, LLMmap is designed to be robust across different application layers, allowing it to identify LLM versions--whether open-source or proprietary--from various vendors, operating under various unknown system prompts, stochastic sampling hyperparameters, and even complex generation frameworks such as RAG or Chain-of-Thought. We discuss potential mitigations and demonstrate that, against resourceful adversaries, effective countermeasures may be challenging or even unrealizable.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑