Ethicist：通过损失平滑软提示与校准置信度估计实现定向训练数据提取
Ethicist: Targeted Training Data Extraction Through Loss Smoothed Soft Prompting and Calibrated Confidence Estimation
AI总结：
本文提出 Ethicist 方法，通过损失平滑软提示和校准置信度估计实现定向训练数据提取，显著提升公开基准上的提取性能，并分析了多种影响因素。
摘要：
大型预训练语言模型在众多任务上取得了令人瞩目的成果。然而，近期研究指出，预训练语言模型可能记忆了其训练数据中相当大的一部分，从而带来信息泄露的隐私风险。本文提出了一种名为 Ethicist 的方法，通过损失平滑软提示和校准置信度估计实现定向训练数据提取，研究在给定前缀时如何恢复训练数据中的后缀。为了激发被攻击模型的记忆行为，我们在保持模型固定的同时调整软提示嵌入。我们进一步提出了一种平滑损失，用于平滑后缀 token 的损失分布，使采样到正确后缀更加容易。为了从一组采样得到的后缀中选出最可能的后缀并估计预测置信度，我们提出了一种校准置信度估计方法，该方法利用局部估计对生成后缀的置信度进行归一化。我们证明 Ethicist 在最近提出的公开基准上显著提升了提取性能。我们还研究了影响数据提取性能的若干因素，包括解码策略、模型规模、前缀长度和后缀长度。我们的代码可在 https://github.com/thu-coai/Targeted-Data-Extraction 获取。
英文摘要：
Large pre-trained language models achieve impressive results across many tasks. However, recent works point out that pre-trained language models may memorize a considerable fraction of their training data, leading to the privacy risk of information leakage. In this paper, we propose a method named Ethicist for targeted training data extraction through loss smoothed soft prompting and calibrated confidence estimation, investigating how to recover the suffix in the training data when given a prefix. To elicit memorization in the attacked model, we tune soft prompt embeddings while keeping the model fixed. We further propose a smoothing loss that smooths the loss distribution of the suffix tokens to make it easier to sample the correct suffix. In order to select the most probable suffix from a collection of sampled suffixes and estimate the prediction confidence, we propose a calibrated confidence estimation method, which normalizes the confidence of the generated suffixes with a local estimation. We show that Ethicist significantly improves the extraction performance on a recently proposed public benchmark. We also investigate several factors influencing the data extraction performance, including decoding strategy, model scale, prefix length, and suffix length. Our code is available at https://github.com/thu-coai/Targeted-Data-Extraction.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
