arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.14549cs.CVcs.LGq-bio.NC
收藏

揭示潜在记忆:评估前沿AI模型中的数据泄露与记忆模式

Uncovering Latent Memories: Assessing Data Leakage and Memorization Patterns in Frontier AI Models

Stanford University(斯坦福大学)

Sunny Duan, Mikail Khona, Abhiram Iyer, Rylan Schaeffer, Ila R Fiete

AI总结:

该论文研究了前沿AI模型训练过程中的记忆演变,发现了“潜在记忆”现象,并开发了基于交叉熵损失的诊断测试以高准确率揭示隐藏的记忆序列,为数据隐私保护提供了新方法。

摘要:

前沿AI系统正在对社会产生变革性影响,但这种益处并非没有代价:在包含个人和私有数据的网络规模数据集上训练的模型,引发了对数据隐私和安全的深刻担忧。语言模型在包含潜在敏感或专有信息的大规模语料库上进行训练,而数据泄露(即模型响应泄露此类信息的片段)的风险仍未得到充分理解。先前的工作研究了驱动记忆的因素,并发现序列复杂度和重复次数驱动记忆。在这里,我们关注训练过程中记忆的演变。我们首先重现了以下发现:记忆一个序列的概率与其在数据中出现的次数呈对数缩放关系。接下来,我们展示了在首次遭遇后表面上未被记忆的序列,即使没有后续遭遇,也可以在整个训练过程中被“揭示”,我们将这一现象称为“潜在记忆”。潜在记忆的存在对数据隐私构成了挑战,因为被记忆的序列可能在模型的最终检查点中被隐藏,但仍然很容易恢复。为此,我们开发了一种依赖交叉熵损失的诊断测试,以高准确率揭示潜在记忆序列。

英文摘要:

Frontier AI systems are making transformative impacts across society, but such benefits are not without costs: models trained on web-scale datasets containing personal and private data raise profound concerns about data privacy and security. Language models are trained on extensive corpora including potentially sensitive or proprietary information, and the risk of data leakage - where the model response reveals pieces of such information - remains inadequately understood. Prior work has investigated what factors drive memorization and have identified that sequence complexity and the number of repetitions drive memorization. Here, we focus on the evolution of memorization over training. We begin by reproducing findings that the probability of memorizing a sequence scales logarithmically with the number of times it is present in the data. We next show that sequences which are apparently not memorized after the first encounter can be "uncovered" throughout the course of training even without subsequent encounters, a phenomenon we term "latent memorization". The presence of latent memorization presents a challenge for data privacy as memorized sequences may be hidden at the final checkpoint of the model but remain easily recoverable. To this end, we develop a diagnostic test relying on the cross entropy loss to uncover latent memorized sequences with high accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑