为什么我的提示词会泄露？剖析定制化大型语言模型中的提示词提取威胁
Why Are My Prompts Leaked? Unraveling Prompt Extraction Threats in Customized Large Language Models
AI总结：
针对定制化大型语言模型（LLMs）的提示词泄露问题，本文分析了提示词记忆机制及影响提取的关键属性，提出基于困惑度和注意力矩阵的假设，发现现有对齐模型仍易受攻击，进而提出防御策略使Llama2-7B和GPT-3.5的提取率大幅下降。
摘要：
大型语言模型（LLMs）参数的急剧增加催生了一种无需微调的下游定制新研究方向——通过提示词（即任务描述）实现。尽管这些基于提示词的服务（如OpenAI的GPTs）在许多业务中发挥着重要作用，但人们对提示词泄露的担忧日益加剧，这会损害这些服务的知识产权并引发下游攻击。本文分析了提示词泄露的潜在机制（我们称之为提示词记忆），并制定了相应的防御策略。通过探索提示词提取中的缩放定律，我们分析了影响提示词提取的关键属性，包括模型规模、提示词长度以及提示词类型。然后我们提出两个假设来解释LLMs如何暴露其提示词：第一个归因于困惑度（即LLMs对文本的熟悉度），第二个基于注意力矩阵中直接的token翻译路径。为了防御此类威胁，我们研究了对齐是否能削弱提示词提取。我们发现，当前的LLMs，即使是像GPT-4这样具有安全对齐的模型，也极易受到提示词提取攻击，即使在最直接的用户攻击下也是如此。因此，我们根据研究结果提出了几种防御策略，这些策略分别使Llama2-7B和GPT-3.5的提示词提取率下降了83.8%和71.0%。源代码可在https://github.com/liangzid/PromptExtractionEval获取。
英文摘要：
The drastic increase of large language models' (LLMs) parameters has led to a new research direction of fine-tuning-free downstream customization by prompts, i.e., task descriptions. While these prompt-based services (e.g. OpenAI's GPTs) play an important role in many businesses, there has emerged growing concerns about the prompt leakage, which undermines the intellectual properties of these services and causes downstream attacks. In this paper, we analyze the underlying mechanism of prompt leakage, which we refer to as prompt memorization, and develop corresponding defending strategies. By exploring the scaling laws in prompt extraction, we analyze key attributes that influence prompt extraction, including model sizes, prompt lengths, as well as the types of prompts. Then we propose two hypotheses that explain how LLMs expose their prompts. The first is attributed to the perplexity, i.e. the familiarity of LLMs to texts, whereas the second is based on the straightforward token translation path in attention matrices. To defend against such threats, we investigate whether alignments can undermine the extraction of prompts. We find that current LLMs, even those with safety alignments like GPT-4, are highly vulnerable to prompt extraction attacks, even under the most straightforward user attacks. Therefore, we put forward several defense strategies with the inspiration of our findings, which achieve 83.8\% and 71.0\% drop in the prompt extraction rate for Llama2-7B and GPT-3.5, respectively. Source code is avaliable at https://github.com/liangzid/PromptExtractionEval.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
