arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.18733cs.LG
收藏

先验意识记忆:一种区分记忆与泛化在大语言模型中的高效度量标准

A Prior-Aware Metric for Efficiently Distinguishing Memorization from Generalization in Large Language Models

Cornell University(康奈尔大学);Boston University(波士顿大学);NVIDIA(英伟达)

Trishita Tiwari, Ari Trachtenberg, G. Edward Suh

AI总结:

本文提出Prior-Aware Memorization,一种基于模型先验的轻量级度量标准,用于区分大语言模型中的真实记忆与统计上常见序列,揭示低频率不足以证明记忆,强调需考虑模型先验以评估数据泄露风险。

摘要:

训练数据泄露从大型语言模型(LLMs)中引发严重的隐私、安全和版权合规问题。评估这一风险的核心挑战是区分真实训练数据的记忆与统计上常见的序列生成。现有的测量记忆的方法往往将这些现象混为一谈,将输出标记为记忆即使它们源于对常见模式的泛化。Counterfactual Memorization提供了一个原理性解决方案,通过比较训练时包含和不包含目标序列的模型进行比较,但其依赖于重新训练多个基线模型使其计算成本高昂且在大规模上不切实际。 本文介绍Prior-Aware Memorization,一种理论上站得住脚、轻量级且训练免费的标准,用于识别LLMs中的真实记忆。关键思想是评估候选后缀是否强烈关联于其特定的训练前缀,或者是否在许多无关提示中以高概率出现,由于统计上的常见性。 我们在两个预训练模型(LLaMA和OPT)的训练语料库中的文本上评估这一指标,使用长序列(以模拟版权风险)和命名实体(以模拟PII泄露)。我们的结果表明,之前被标记为记忆的序列中有55%至90%实际上是统计上常见的。相似的发现适用于SATML训练数据提取挑战数据集,在该数据集中,尽管在训练数据中只出现一次,大约40%的序列表现出常见模式行为。这些结果表明,低频率本身不足以作为记忆的证据,并突显了在评估泄露时考虑模型先验的重要性。

英文摘要:

Training data leakage from Large Language Models (LLMs) raises serious concerns related to privacy, security, and copyright compliance. A central challenge in assessing this risk is distinguishing prefix-specific memorization of training data from the generation of statistically common sequences. Existing approaches to measuring memorization often conflate these phenomena, labeling outputs as memorized even when they arise from generalization over common patterns. Counterfactual memorization and other related metrics \citep{zhang2023counterfactual, wang2025generalization, lesci2024causal} provide principled solutions, however, their reliance on retraining multiple baseline models or parsing through the training data makes them computationally impractical at scale. This work introduces \emph{Prior-Aware memorization}, a theoretically grounded, lightweight and training-free criterion for identifying prefix-specific memorization in LLMs. The key idea is to evaluate whether a candidate suffix is strongly associated with its specific training prefix or whether it appears with high probability across many IID sampled sequences from the training data distribution due to statistical commonality. We correlate our metric with counterfactual memorization, and also evaluate it on the training corpora of two pre-trained models, LLaMA and OPT. Our results show that between 55\% and 90\% of sequences previously labeled as memorized fail our criterion and are consistent with statistical commonality

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑