从语言模型中有效提取提示词
Effective Prompt Extraction from Language Models
AI总结：
本文提出系统衡量提示提取攻击有效性的框架，实验表明简单文本攻击能高概率揭示模型提示，并能区分真实提示与幻觉，验证了Claude 3和ChatGPT等系统提示可被提取。
摘要：
大型语言模型生成的文本通常通过提示来控制，其中在用户查询前添加的提示会引导模型的输出。公司用于引导其模型的提示通常被视为秘密，需要对发起查询的用户隐藏。它们甚至被当作商品在市场上买卖。然而，轶事报告显示，对抗性用户利用提示提取攻击来恢复这些提示。在本文中，我们提出了一个系统衡量这些攻击有效性的框架。在使用3种不同来源的提示和11个底层大型语言模型的实验中，我们发现简单的基于文本的攻击实际上能够以高概率揭示提示。我们的框架能够高精度地判断提取出的提示是真实的秘密提示，还是模型幻觉。从Claude 3和ChatGPT等真实系统中提取提示进一步表明，尽管已有防御措施，系统提示仍可能被对手揭示。
英文摘要：
The text generated by large language models is commonly controlled by prompting, where a prompt prepended to a user's query guides the model's output. The prompts used by companies to guide their models are often treated as secrets, to be hidden from the user making the query. They have even been treated as commodities to be bought and sold on marketplaces. However, anecdotal reports have shown adversarial users employing prompt extraction attacks to recover these prompts. In this paper, we present a framework for systematically measuring the effectiveness of these attacks. In experiments with 3 different sources of prompts and 11 underlying large language models, we find that simple text-based attacks can in fact reveal prompts with high probability. Our framework determines with high precision whether an extracted prompt is the actual secret prompt, rather than a model hallucination. Prompt extraction from real systems such as Claude 3 and ChatGPT further suggest that system prompts can be revealed by an adversary despite existing defenses in place.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
