从威胁到工具：利用拒识感知注入攻击实现安全对齐
From Threat to Tool: Leveraging Refusal-Aware Injection Attacks for Safety Alignment
AI总结：
本研究提出无需训练的拒识感知自适应注入（RAAI）框架，将LLM攻击技术转化为合成数据生成工具，用其生成的数据微调模型可提升安全鲁棒性且保留通用能力，为LLM安全对齐提供了高效新路径。
摘要：
大语言模型（LLM）的安全对齐通常需要大量人工标注的偏好数据，这一过程成本高昂且耗时。尽管合成数据提供了颇具前景的替代方案，但现有方法往往依赖复杂的迭代提示工程或辅助模型。为解决该问题，我们提出拒识感知自适应注入（Refusal-Aware Adaptive Injection，RAAI）框架，这是一种无需训练、模型无关的简洁框架，可重新利用LLM攻击技术。RAAI通过检测内部拒绝信号，自适应注入预定义短语，诱导模型生成有害但流畅的补全内容。实验表明，RAAI可有效破解LLM的安全防护，在四个基准测试上，有害响应率从基线的2.15%平均提升至最高61.04%。关键的是，使用RAAI生成的合成数据微调LLM，可提升模型对有害提示的鲁棒性，同时保留其在MMLU、ARC等标准任务上的通用能力。本研究表明，LLM攻击方法可被重构为可扩展、可控的安全对齐实用工具。
英文摘要：
Safely aligning large language models (LLMs) often demands extensive human-labeled preference data, a process that's both costly and time-consuming. While synthetic data offers a promising alternative, current methods frequently rely on complex iterative prompting or auxiliary models. To address this, we introduce Refusal-Aware Adaptive Injection (RAAI), a straightforward, training-free, and model-agnostic framework that repurposes LLM attack techniques. RAAI works by detecting internal refusal signals and adaptively injecting predefined phrases to elicit harmful, yet fluent, completions. Our experiments show RAAI effectively jailbreaks LLMs, increasing the harmful response rate from a baseline of 2.15% to up to 61.04% on average across four benchmarks. Crucially, fine-tuning LLMs with the synthetic data generated by RAAI improves model robustness against harmful prompts while preserving general capabilities on standard tasks like MMLU and ARC. This work highlights how LLM attack methodologies can be reframed as practical tools for scalable and controllable safety alignment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
