Gandalf the Red:面向大语言模型的自适应安全防护
Gandalf the Red: Adaptive Security for LLMs
AI总结:
本文提出D-SEC威胁模型和Gandalf众包红队平台,通过279k攻击数据集分析,发现LLM内置防御会损害可用性,并验证了受限领域、纵深防御和自适应防御的有效性。
摘要:
当前针对大语言模型(LLM)应用中提示词攻击防御的评估,往往忽视两个关键因素:对抗行为的动态性,以及严格防御措施对合法用户造成的可用性损失。我们提出D-SEC(动态安全-效用威胁模型),该模型明确区分攻击者与合法用户,建模多步交互,并以可优化的形式表达安全-效用权衡。为进一步解决现有评估的不足,我们引入Gandalf——一个众包、游戏化的红队平台,旨在生成真实、自适应的攻击。利用Gandalf,我们收集并发布了一个包含279,000次提示词攻击的数据集。结合良性用户数据,我们的分析揭示了安全与效用之间的相互作用,表明集成在LLM中的防御(如系统提示)即使在不阻止请求的情况下也可能降低可用性。我们证明,受限应用领域、纵深防御和自适应防御是构建安全且有用的LLM应用的有效策略。
英文摘要:
Current evaluations of defenses against prompt attacks in large language model (LLM) applications often overlook two critical factors: the dynamic nature of adversarial behavior and the usability penalties imposed on legitimate users by restrictive defenses. We propose D-SEC (Dynamic Security Utility Threat Model), which explicitly separates attackers from legitimate users, models multi-step interactions, and expresses the security-utility in an optimizable form. We further address the shortcomings in existing evaluations by introducing Gandalf, a crowd-sourced, gamified red-teaming platform designed to generate realistic, adaptive attack. Using Gandalf, we collect and release a dataset of 279k prompt attacks. Complemented by benign user data, our analysis reveals the interplay between security and utility, showing that defenses integrated in the LLM (e.g., system prompts) can degrade usability even without blocking requests. We demonstrate that restricted application domains, defense-in-depth, and adaptive defenses are effective strategies for building secure and useful LLM applications.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。