AgentAlign：在从信息型到智能体大型语言模型的转变中导航安全对齐
AgentAlign: Navigating Safety Alignment in the Shift from Informative to Agentic Large Language Models
AI总结：
本文提出AgentAlign框架，利用抽象行为链在模拟环境中合成安全对齐数据，有效提升开源LLM智能体的安全性（提升35.8%至79.5%）并保持其有用性。
摘要：
智能体能力的获取已将LLMs从“知识提供者”转变为“行动执行者”，这一趋势虽然扩展了LLMs的能力边界，但也显著增加了其被恶意利用的风险。先前的工作表明，当前的基于LLM的智能体即使在没有受到攻击的情况下也会执行大量恶意任务，这表明在后训练阶段缺乏智能体使用安全对齐。为了解决这一差距，我们提出了AgentAlign，这是一个新颖的框架，利用抽象行为链作为安全对齐数据合成的媒介。通过在具有多样化工具实例的模拟环境中实例化这些行为链，我们的框架能够生成高度真实且可执行的指令，同时捕捉复杂的多步骤动态。该框架进一步通过对行为链的非恶意解释按比例合成良性指令来确保模型实用性，精确校准有用性与无害性之间的边界。在AgentHarm上的评估结果表明，使用我们的方法对三个开源模型系列进行微调，可显著提高其安全性（提升35.8%至79.5%），同时对有用性的影响极小甚至有积极提升，优于各种提示方法。数据集和代码均已开源。
英文摘要：
The acquisition of agentic capabilities has transformed LLMs from "knowledge providers" to "action executors", a trend that while expanding LLMs' capability boundaries, significantly increases their susceptibility to malicious use. Previous work has shown that current LLM-based agents execute numerous malicious tasks even without being attacked, indicating a deficiency in agentic use safety alignment during the post-training phase. To address this gap, we propose AgentAlign, a novel framework that leverages abstract behavior chains as a medium for safety alignment data synthesis. By instantiating these behavior chains in simulated environments with diverse tool instances, our framework enables the generation of highly authentic and executable instructions while capturing complex multi-step dynamics. The framework further ensures model utility by proportionally synthesizing benign instructions through non-malicious interpretations of behavior chains, precisely calibrating the boundary between helpfulness and harmlessness. Evaluation results on AgentHarm demonstrate that fine-tuning three families of open-source models using our method substantially improves their safety (35.8% to 79.5% improvement) while minimally impacting or even positively enhancing their helpfulness, outperforming various prompting methods. The dataset and code have both been open-sourced.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
