arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2504.09604cs.LGcs.AIcs.CR
收藏

缓解多示例劫持

Mitigating Many-Shot Jailbreaking

Christopher M. Ackerman, Nina Panickssery

AI总结:

本文研究了通过微调和输入净化缓解多示例劫持攻击的有效性,发现组合方法显著降低攻击效果,同时保持模型在正常上下文学习中的性能。

摘要:

多示例劫持(MSJ)是一种对抗性技术,利用现代大语言模型(LLM)的长上下文窗口,通过在提示中包含大量“伪造”助手不当回应的例子来绕过模型安全训练。本文探讨了不同微调和输入净化方法在缓解MSJ攻击中的有效性,单独和组合使用。研究发现每种方法都有逐步缓解效果,组合技术显著降低MSJ攻击效果,同时保持模型在良性上下文学习和对话任务中的性能。我们建议如果将该方法纳入模型安全后训练,可以有效缓解此漏洞。

英文摘要:

Many-shot jailbreaking (MSJ) is an adversarial technique that exploits the long context windows of modern LLMs to circumvent model safety training by including in the prompt many examples of a "fake" assistant responding inappropriately before the final request. With enough examples, the model's in-context learning abilities override its safety training, and it responds as if it were the "fake" assistant. In this work, we probe the effectiveness of different fine-tuning and input sanitization approaches on mitigating MSJ attacks, alone and in combination. We find incremental mitigation effectiveness for each, and show that the combined techniques significantly reduce the effectiveness of MSJ attacks, while retaining model performance in benign in-context learning and conversational tasks. We suggest that our approach could meaningfully ameliorate this vulnerability if incorporated into model safety post-training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑