自适应指令生成用于自动化大语言模型红队测试
Adaptive Instruction Composition for Automated LLM Red-Teaming
AI总结：
本文提出自适应指令生成框架，通过强化学习平衡探索与利用，提升红队测试的攻击效果与多样性，优于随机组合和现有方法。
摘要：
许多大语言模型红队测试方法利用攻击者LLM发现 jailbreaks。一些方法通过试错让攻击者识别有效策略，导致成功范围有限。另一种方法通过结合众包有害查询和战术生成指令，但随机组合限制了效果。本文介绍自适应指令生成框架，根据训练好的适应机制组合众包文本，以共同优化效果与多样性。我们使用强化学习在指令的组合空间中平衡探索与利用，引导攻击者生成针对目标漏洞的多样化输出。我们证明，我们的方法在效果和多样性指标上显著优于随机组合，即使在模型迁移下也是如此。此外，我们显示它在Harmbench上超越了多种近期自适应方法。我们采用轻量级神经上下文带轮，适应对比嵌入输入，并提供消融研究，表明对比预训练使网络能够快速泛化并扩展到大规模空间，因为它在学习过程中。
英文摘要：
Many approaches to LLM red-teaming leverage an attacker LLM to discover jailbreaks against a target. Several of them task the attacker with identifying effective strategies through trial and error, resulting in a semantically limited range of successes. Another approach discovers diverse attacks by combining crowdsourced harmful queries and tactics into instructions for the attacker, but does so at random, limiting effectiveness. This article introduces a novel framework, Adaptive Instruction Composition, that combines crowdsourced texts according to an adaptive mechanism trained to jointly optimize effectiveness with diversity. We use reinforcement learning to balance exploration with exploitation in a combinatorial space of instructions to guide the attacker toward diverse generations tailored to target vulnerabilities. We demonstrate that our approach substantially outperforms random combination on a set of effectiveness and diversity metrics, even under model transfer. Further, we show that it surpasses a host of recent adaptive approaches on Harmbench. We employ a lightweight neural contextual bandit that adapts to contrastive embedding inputs, and provide ablations suggesting that the contrastive pretraining enables the network to rapidly generalize and scale to the massive space as it learns.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
