Alpaca 对抗 Vicuna：利用大语言模型（LLMs）揭示大语言模型的记忆问题
Alpaca against Vicuna: Using LLMs to Uncover Memorization of LLMs
AI总结：
本文提出黑盒提示优化方法，通过攻击者LLM智能体以迭代拒绝采样生成低训练数据重叠、高受害模型输出重叠的指令提示，揭示更高程度的LLM记忆问题，为自动化攻击研究提供新方向。
摘要：
本文提出一种黑盒提示优化方法，通过攻击者大语言模型（LLM）智能体揭示受害智能体更高程度的记忆问题，相比直接用训练数据提示目标模型（当前量化LLM记忆的主流方法）效果更显著。我们采用迭代拒绝采样优化过程，寻找具备两大特征的基于指令的提示：（1）与训练数据重叠最小，避免直接向模型呈现解决方案；（2）受害模型输出与训练数据重叠最大，旨在诱导受害模型吐出训练数据。我们观察到，与基线前缀-后缀测量相比，基于指令的提示生成的输出与训练数据的重叠度高出23.7%。我们的研究结果表明：（1）指令调优模型暴露预训练数据的程度不亚于其基础模型，甚至可能更高；（2）原始训练数据之外的上下文也可能导致数据泄露；（3）使用其他LLM提出的指令可开辟自动化攻击新途径，需进一步研究和探索。代码可在https://github.com/Alymostafa/Instruction_based_attack获取。
英文摘要：
In this paper, we introduce a black-box prompt optimization method that uses an attacker LLM agent to uncover higher levels of memorization in a victim agent, compared to what is revealed by prompting the target model with the training data directly, which is the dominant approach of quantifying memorization in LLMs. We use an iterative rejection-sampling optimization process to find instruction-based prompts with two main characteristics: (1) minimal overlap with the training data to avoid presenting the solution directly to the model, and (2) maximal overlap between the victim model's output and the training data, aiming to induce the victim to spit out training data. We observe that our instruction-based prompts generate outputs with 23.7% higher overlap with training data compared to the baseline prefix-suffix measurements. Our findings show that (1) instruction-tuned models can expose pre-training data as much as their base-models, if not more so, (2) contexts other than the original training data can lead to leakage, and (3) using instructions proposed by other LLMs can open a new avenue of automated attacks that we should further study and explore. The code can be found at https://github.com/Alymostafa/Instruction_based_attack .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
