召魔缚魔：大语言模型红队测试的扎根理论
Summon a Demon and Bind it: A Grounded Theory of LLM Red Teaming
AI总结：
该研究通过定性访谈数十名从业者，界定了LLM红队测试为非恶意的极限探寻类手动活动，归纳其动机与12种策略、35种攻击技术，构建了LLM红队测试的扎根理论。
摘要：
通过攻击大语言模型（Large Language Models, LLMs）刻意生成异常输出，是一种新兴的人类活动。本文基于广泛多样的证据，详尽阐述了人们实施此类攻击的方式与原因，并对LLM红队测试给出了定义。我们采用正式的定性研究方法，访谈了数十名背景各异的从业者，他们均是这项旨在引发LLM失效的新兴工作的贡献者。本研究聚焦三大问题：界定LLM红队测试、揭示开展该活动的动机与目标，以及归纳人们攻击LLM时使用的策略。基于研究数据，LLM红队测试被定义为一种探寻极限、非恶意的手动活动，高度依赖团队协作与“炼金术士思维”。其内在动机极强，源于好奇心、趣味性，在一定程度上也出于对部署LLM可能带来的各类危害的担忧。我们识别出包含12种策略、35种不同技术的LLM攻击分类体系。这些发现构成了一套关于人们攻击大语言模型的方式与动因的综合扎根理论——LLM红队测试。
英文摘要：
Engaging in the deliberate generation of abnormal outputs from Large Language Models (LLMs) by attacking them is a novel human activity. This paper presents a thorough exposition of how and why people perform such attacks, defining LLM red-teaming based on extensive and diverse evidence. Using a formal qualitative methodology, we interviewed dozens of practitioners from a broad range of backgrounds, all contributors to this novel work of attempting to cause LLMs to fail. We focused on the research questions of defining LLM red teaming, uncovering the motivations and goals for performing the activity, and characterizing the strategies people use when attacking LLMs. Based on the data, LLM red teaming is defined as a limit-seeking, non-malicious, manual activity, which depends highly on a team-effort and an alchemist mindset. It is highly intrinsically motivated by curiosity, fun, and to some degrees by concerns for various harms of deploying LLMs. We identify a taxonomy of 12 strategies and 35 different techniques of attacking LLMs. These findings are presented as a comprehensive grounded theory of how and why people attack large language models: LLM red teaming.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
