用于区分大语言模型的20个问题游戏
The 20 questions game to distinguish large language models
AI总结:
本研究类比20个问题游戏,提出黑盒场景下区分大语言模型的方法,通过形式化问题、建立基线并设计两种提问启发式,以更少问题实现22个LLM的区分,具备高隐蔽性,可服务于模型审计与版权核查。
摘要:
类比20个问题游戏,我们提出了一种方法,用于判断处于黑盒场景下的两个大语言模型(LLM)是否为同一模型。目标是使用少量(良性的)二元问题,通常不超过20个。我们对该问题进行了形式化定义,首先通过从已知基准数据集中随机选择问题建立了基线,在20个问题内达到了近100%的准确率。在给出该问题的最优边界后,我们提出了两种有效的提问启发式方法,在完成相同任务时所需问题数量仅为前者的一半,可区分22个LLM。这些方法在隐蔽性方面具有显著优势,因此对面临模型泄露怀疑的审计人员或版权所有者具有实用价值。
英文摘要:
In a parallel with the 20 questions game, we present a method to determine whether two large language models (LLMs), placed in a black-box context, are the same or not. The goal is to use a small set of (benign) binary questions, typically under 20. We formalize the problem and first establish a baseline using a random selection of questions from known benchmark datasets, achieving an accuracy of nearly 100% within 20 questions. After showing optimal bounds for this problem, we introduce two effective questioning heuristics able to discriminate 22 LLMs by using half as many questions for the same task. These methods offer significant advantages in terms of stealth and are thus of interest to auditors or copyright owners facing suspicions of model leaks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。