RUBAS: 基于评分标准的强化学习用于智能体安全
RUBAS: Rubric-Based Reinforcement Learning for Agent Safety
AI总结：
提出RUBAS框架，通过将智能体行为分解为四个维度的评分标准提供细粒度奖励，利用强化学习在保证任务完成的同时提升工具使用安全性。
摘要：
LLM进化为工具型智能体带来了与真实世界执行相关的新安全挑战，而非简单的文本生成。现有的对齐方法通常依赖粗略的拒绝信号或静态监督，难以在多样化的智能体风险中平衡安全性与有用的工具执行。我们提出了RUBAS，一种基于评分标准的强化学习框架用于智能体安全。RUBAS将智能体行为分解为四个维度：工具使用安全性、参数安全性、响应安全性和有用性。这些结构化的评分标准在完整的智能体轨迹上提供细粒度且可解释的奖励，使强化学习能够在保持任务完成的同时优化安全工具使用。在多个智能体安全基准和模型上的大量实验表明，RUBAS相比标准对齐基线提高了安全性，减少了基于工具的幻觉，并保持了竞争性的实用性。我们的结果表明，多维评分标准奖励为在安全关键的工具使用环境中对齐LLM智能体提供了有效的训练信号。
英文摘要：
The evolution of LLMs into tool-enabled agents creates a new class of safety challenges associated with real-world execution rather than simple text generation. Existing alignment methods often rely on coarse refusal signals or static supervision, making it difficult to balance safety with useful tool execution across diverse agentic risks. We introduce RUBAS, a rubric-based reinforcement learning framework for agent safety. RUBAS decomposes agent behavior into four dimensions: tool-use safety, argument safety, response safety, and helpfulness. These structured rubrics provide fine-grained and interpretable rewards over complete agent trajectories, enabling reinforcement learning to optimize safe tool use while preserving task completion. Extensive experiments across multiple agent safety benchmarks and models show that RUBAS improves safety over standard alignment baselines, reduces tool-grounded hallucinations, and maintains competitive utility. Our results suggest that multi-dimensional rubric rewards provide an effective training signal for aligning LLM agents in safety-critical tool-use settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
