评估大型语言模型对提示注入的指令遵循鲁棒性
Evaluating the Instruction-Following Robustness of Large Language Models to Prompt Injection
AI总结：
本文构建了一个基准来评估大型语言模型对提示注入攻击的指令遵循鲁棒性，通过大量实验揭示了模型在区分注入指令与原始指令方面的重大漏洞，并指出未来应重点提升模型对提示的整体理解与指令辨别能力。
摘要：
大型语言模型（LLMs）在指令遵循方面展现出了卓越的能力，在各类应用中变得日益重要。然而，这种能力也带来了提示注入攻击的风险，攻击者将指令注入到LLMs的输入中以诱发不良行为或内容。了解LLMs对此类攻击的鲁棒性对于其安全实施至关重要。在这项工作中，我们建立了一个基准来评估遵循指令的LLMs对提示注入攻击的鲁棒性。我们的目标是确定LLMs在多大程度上会受到注入指令的影响，以及它们区分这些注入指令与原始目标指令的能力。通过对领先的指令遵循LLMs进行大量实验，我们揭示了它们在此类攻击鲁棒性方面的重大漏洞。我们的结果表明，一些模型过度调整为遵循提示中的任何嵌入指令，过度关注提示的后半部分而没有完全理解整个上下文。相比之下，对上下文和指令遵循能力把握更好的模型可能更容易受到注入指令的破坏。这凸显了将重点从仅仅增强LLMs的指令遵循能力转移到提高其对提示的整体理解以及对适合遵循的指令的辨别能力上的必要性。我们希望我们的深入分析能为这些漏洞的根本原因提供见解，有助于未来解决方案的开发。代码和数据可在 https://github.com/Leezekun/instruction-following-robustness-eval 获取。
英文摘要：
Large Language Models (LLMs) have demonstrated exceptional proficiency in instruction-following, becoming increasingly crucial across various applications. However, this capability brings with it the risk of prompt injection attacks, where attackers inject instructions into LLMs' input to elicit undesirable actions or content. Understanding the robustness of LLMs against such attacks is vital for their safe implementation. In this work, we establish a benchmark to evaluate the robustness of instruction-following LLMs against prompt injection attacks. Our objective is to determine the extent to which LLMs can be influenced by injected instructions and their ability to differentiate between these injected and original target instructions. Through extensive experiments with leading instruction-following LLMs, we uncover significant vulnerabilities in their robustness to such attacks. Our results indicate that some models are overly tuned to follow any embedded instructions in the prompt, overly focusing on the latter parts of the prompt without fully grasping the entire context. By contrast, models with a better grasp of the context and instruction-following capabilities will potentially be more susceptible to compromise by injected instructions. This underscores the need to shift the focus from merely enhancing LLMs' instruction-following capabilities to improving their overall comprehension of prompts and discernment of instructions that are appropriate to follow. We hope our in-depth analysis offers insights into the underlying causes of these vulnerabilities, aiding in the development of future solutions. Code and data are available at https://github.com/Leezekun/instruction-following-robustness-eval
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
