arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2606.06286cs.CLcs.AI
收藏

LLMs 可能泄露训练数据,但它们愿意吗?一种基于倾向性的 LLM 记忆评估

LLMs Can Leak Training Data But Do They Want To? Evaluating Memorization Propensity in LLMs

University of Southern Denmark(南部丹麦大学)

Gianluca Barmina, Peter Schneider-Kamp, Lukas Galke Poech

AI总结:

提出 PropMe 框架,通过对比前缀攻击与非对抗评估,揭示 LLM 在非对抗设置下很少泄露训练数据,并引入 SimpleTrace 流水线进行归因和度量。

摘要:

大型语言模型可以重现训练数据,但现有的记忆评估大多衡量模型是否可以被强制这样做,而不是在正常使用下是否会这样做。我们引入了 PropMe,一个基于倾向性的记忆评估框架,对比了基于前缀的能力攻击与非对抗性评估。我们提出了一种度量转换方法,应用于现有函数,可以创建倾向性度量。我们进一步引入了 SimpleTrace,一个基于 infini-gram 的轻量级追踪流水线,能够确定性地将模型生成归因于大规模训练语料库,并计算逐字、近逐字和倾向性转换的记忆度量。评估两个完全开放的模型:Comma 和 DFM Decoder,在两个数据集:Common Pile 和 Dynaword,以及两种语言上,我们发现能力与倾向性之间存在一致差距:前缀攻击比通用或数据集特定提示引发更强的记忆信号,而倾向性得分总体保持较低。因此,模型在直接诱导时可以泄露训练数据,但在更常见的非对抗设置中很少这样做。我们还发现,从 Comma 持续预训练的 DFM Decoder 对 Common Pile 表现出降低的记忆和记忆倾向性,证实当后续训练强调部分不同数据时,记忆能力可能下降。我们的结果表明,并鼓励,记忆审计应同时报告最坏情况下的可提取性和普通泄露倾向性,以便更全面地理解这一现象。

英文摘要:

Large language models can reproduce training data, but memorization evaluations mostly measure whether models can be made to do so, not whether they do so without adversarial prompting. We introduce PropMe, a framework that pairs a capability evaluation (prefix attacks) with non-adversarial propensity evaluation, and a propensity metric to relate the two. We check every generation against the model's full indexed training corpus with SimpleTrace, an open-source batch implementation of OLMoTrace span matching. We evaluate Comma, its continually pre-trained descendant DFM Decoder, and OLMo3 32B on Common Pile, Dynaword, and Dolma3, with four non-adversarial settings ranging from no prompt to sentences from unseen in-domain sources. Our results show that natural sentence prompts reproduce far less training text than prefix attacks. Generation with no prompt or a single common word, however, reproduces about as much as prefix attacks, or more, in three out of four model-corpus pairs; most of this text is duplicated licenses, code, and templates. In our setting, continual pre-training reduces long verbatim reproduction of the original corpus only modestly. More broadly, we argue that by testing both capability and propensity, we can attain a more comprehensive picture of a model's memorization behavior.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑