arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.19482cs.LG
收藏

通过概率提取衡量语言模型中的记忆情况

Measuring memorization in language models via probabilistic extraction

Google DeepMind(谷歌DeepMind);Northeastern(东北大学);Microsoft Research(微软研究院);Stanford(斯坦福大学)

Jamie Hayes, Marika Swanberg, Harsh Chaudhari, Itay Yona, Ilia Shumailov, Milad Nasr, Christopher A. Choquette-Choo, Katherine Lee, A. Feder Cooper

AI总结:

针对传统可发现提取方法未考虑非贪心采样非确定性、衡量记忆情况不可靠的问题,提出无需额外成本的概率可发现提取方法,经多场景验证能提供更细致的提取风险信息。

摘要:

大语言模型(LLM)容易记忆训练数据,这引发了人们对其在生成阶段可能提取敏感信息的担忧。可发现提取是衡量这一问题最常用的方法:将一个训练样本拆分为前缀和后缀,然后用前缀提示LLM,若LLM通过贪心采样生成了匹配的后缀,则认为该样本是可提取的。该定义会针对单次查询得出提取是否成功的二元判定。我们发现,尽管该定义计算高效,但并不可靠,因为它未考虑更符合实际的(非贪心)采样方案中存在的非确定性——在这类采样方案下,LLM会针对同一提示生成一系列不同输出。我们提出了概率可发现提取方法,该方法无需额外成本,通过考虑多次查询来量化提取目标序列的概率,从而对可发现提取的定义进行了放宽。我们在不同模型、采样方案和训练数据重复次数下评估了我们的概率度量方法,发现与传统的可发现提取相比,该度量能提供关于提取风险的更细致信息。

英文摘要:

Large language models (LLMs) are susceptible to memorizing training data, raising concerns about the potential extraction of sensitive information at generation time. Discoverable extraction is the most common method for measuring this issue: split a training example into a prefix and suffix, then prompt the LLM with the prefix, and deem the example extractable if the LLM generates the matching suffix using greedy sampling. This definition yields a yes-or-no determination of whether extraction was successful with respect to a single query. Though efficient to compute, we show that this definition is unreliable because it does not account for non-determinism present in more realistic (non-greedy) sampling schemes, for which LLMs produce a range of outputs for the same prompt. We introduce probabilistic discoverable extraction, which, without additional cost, relaxes discoverable extraction by considering multiple queries to quantify the probability of extracting a target sequence. We evaluate our probabilistic measure across different models, sampling schemes, and training-data repetitions, and find that this measure provides more nuanced information about extraction risk compared to traditional discoverable extraction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑