HarDBench: 面向安全人机协作写作的基于草稿的合著越狱攻击基准
HarDBench: A Benchmark for Draft-Based Co-Authoring Jailbreak Attacks for Safe Human-LLM Collaborative Writing
AI总结：
提出HarDBench基准，评估大语言模型在协作写作中面对恶意草稿填充的越狱攻击的鲁棒性，并通过偏好优化实现安全-效用平衡的对齐方法。
摘要：
大语言模型（LLMs）越来越多地被用作协作写作中的合著者，用户从粗略草稿开始，依赖LLMs完成、修改和优化其内容。然而，这种能力带来了严重的安全风险：恶意用户可能通过用危险内容填充不完整草稿来越狱模型，迫使其生成有害输出。在本文中，我们识别了当前LLMs对此类基于草稿的合著越狱攻击的脆弱性，并引入了HarDBench，一个系统性的基准，旨在评估LLMs对此新兴威胁的鲁棒性。HarDBench涵盖一系列高风险领域——包括爆炸物、毒品、武器和网络攻击——并具有现实结构及领域特定提示的特征，以评估模型对有害补全的敏感性。为缓解此风险，我们引入了一种基于偏好优化的安全-效用平衡对齐方法，训练模型拒绝有害补全，同时保持对良性草稿的有用性。实验结果表明，现有LLMs在合著环境中高度脆弱，而我们的对齐方法显著减少了有害输出，且不降低合著能力性能。这为人机协作写作环境中LLMs的评估与对齐提供了新范式。我们的新基准和数据集可在项目页面获取：此 https URL
英文摘要：
Large language models (LLMs) are increasingly used as co-authors in collaborative writing, where users begin with rough drafts and rely on LLMs to complete, revise, and refine their content. However, this capability poses a serious safety risk: malicious users could jailbreak the models-filling incomplete drafts with dangerous content-to force them into generating harmful outputs. In this paper, we identify the vulnerability of current LLMs to such draft-based co-authoring jailbreak attacks and introduce HarDBench, a systematic benchmark designed to evaluate the robustness of LLMs against this emerging threat. HarDBench spans a range of high-risk domains-including Explosives, Drugs, Weapons, and Cyberattacks-and features prompts with realistic structure and domain-specific cues to assess the model susceptibility to harmful completions. To mitigate this risk, we introduce a safety-utility balanced alignment approach based on preference optimization, training models to refuse harmful completions while remaining helpful on benign drafts. Experimental results show that existing LLMs are highly vulnerable in co-authoring contexts and our alignment method significantly reduces harmful outputs without degrading performance on co-authoring capabilities. This presents a new paradigm for evaluating and aligning LLMs in human-LLM collaborative writing settings. Our new benchmark and dataset are available on our project page at https://github.com/untae0122/HarDBench
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
