通过合成数据构建通用智能体系统的基础护栏
Building a Foundational Guardrail for General Agentic Systems via Synthetic Data
AI总结：
针对LLM智能体规划阶段的安全防护缺口，提出基础护栏Safiron，结合跨规划器适配器与紧凑守护模型，并利用合成数据引擎AuraGen生成训练语料，在Pre-Exec Bench基准上实现了稳健的执行前风险检测与泛化。
摘要：
尽管 LLM 智能体可以规划多步任务，但在规划阶段（即任何动作执行之前）进行干预往往是防止危害的最安全方式，因为某些风险一旦实施就会导致严重后果。然而，现有的护栏大多在执行后运行，难以扩展，且在规划层面几乎没有可控监督的空间。为了应对这一挑战，我们指出了当前研究中的三个关键缺口：数据缺口、模型缺口和评估缺口。为了填补数据缺口，我们引入了 AuraGen，这是一个可控引擎，它 (i) 合成良性轨迹，(ii) 注入带有校准难度的类别标记风险，以及 (iii) 通过自动化奖励模型过滤输出，从而为执行前安全生成大规模且可靠的语料库。为了填补守护模型缺口，我们提出了一种基础护栏 Safiron，它结合了跨规划器适配器与紧凑的守护模型。该适配器统一了不同的输入格式，而 Safiron 负责标记风险案例、分配风险类型并生成理由；Safiron 通过两阶段训练和广泛探索的数据配方，实现了跨设置的稳健迁移。为了填补评估缺口，我们发布了 Pre-Exec Bench，这是一个涵盖多样化工具和分支轨迹的现实基准，用于在人工验证的场景中衡量检测、细粒度分类、解释和跨规划器泛化能力。大量实验表明，所提出的护栏在 Pre-Exec Bench 上相对于强基线取得了一致的提升，消融实验进一步提炼出可操作的实践，为更安全的智能体系统提供了实用模板。
英文摘要：
While LLM agents can plan multi-step tasks, intervening at the planning stage-before any action is executed-is often the safest way to prevent harm, since certain risks can lead to severe consequences once carried out. However, existing guardrails mostly operate post-execution, which is difficult to scale and leaves little room for controllable supervision at the plan level. To address this challenge, we highlight three critical gaps in current research: data gap, model gap, and evaluation gap. To close the data gap, we introduce AuraGen, a controllable engine that (i) synthesizes benign trajectories, (ii) injects category-labeled risks with calibrated difficulty, and (iii) filters outputs via an automated reward model, producing large and reliable corpora for pre-execution safety. To close the guardian model gap, we propose a foundational guardrail Safiron, combining a cross-planner adapter with a compact guardian model. The adapter unifies different input formats, while Safiron flags risky cases, assigns risk types, and generates rationales; trained in two stages with a broadly explored data recipe, Safiron achieves robust transfer across settings. To close the evaluation gap, we release Pre-Exec Bench, a realistic benchmark covering diverse tools and branching trajectories, which measures detection, fine-grained categorization, explanation, and cross-planner generalization in human-verified scenarios. Extensive experiments demonstrate consistent gains of the proposed guardrail over strong baselines on Pre-Exec Bench, and ablations further distill actionable practices, providing a practical template for safer agentic systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
