arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.09187cs.LG
收藏

GuardAgent:通过具备知识启用的推理的Guard Agent来保护LLM智能体

GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning

University of Georgia(佐治亚大学);University of Chicago(芝加哥大学);UIUC(伊利诺伊大学香槟分校);University of Texas at Austin(德克萨斯大学奥斯汀分校);University of California, Berkeley(加州大学伯克利分校);Emory University(埃默里大学);Virtue AI

Zhen Xiang, Linzhi Zheng, Yanjie Li, Junyuan Hong, Qinbin Li, Han Xie, Jiawei Zhang, Zidi Xiong, Chulin Xie, Carl Yang, Dawn Song, Bo Li

AI总结:

本文提出首个护栏智能体GuardAgent,通过生成任务计划并映射为护栏代码执行,动态检查目标智能体行动是否满足安全请求,结合LLM推理与记忆模块,在两个新基准上分别取得超98%和83%的护栏准确率。

摘要:

大型语言模型(LLM)智能体的快速发展引发了关于其安全性和保障的新担忧。在本文中,我们提出了GuardAgent,这是第一个通过动态检查目标智能体的行动是否满足给定的安全防护请求来保护目标智能体的护栏智能体。具体而言,GuardAgent首先分析安全防护请求以生成任务计划,然后将该计划映射为护栏代码以供执行。通过执行代码,GuardAgent能够确定性地遵循安全防护请求并保护目标智能体。在这两个步骤中,LLM被用作推理组件,并由从存储先前任务经验的记忆模块中检索到的上下文演示作为补充。此外,我们提出了两个新颖的基准:EICU-AC基准,用于评估医疗智能体的访问控制;以及Mind2Web-SC基准,用于评估Web智能体的安全策略。我们展示了GuardAgent在这两个基准上有效地调节了不同类型智能体的违规行动,护栏准确率分别超过98%和83%。项目页面:https://guardagent.github.io/

英文摘要:

The rapid advancement of large language model (LLM) agents has raised new concerns regarding their safety and security. In this paper, we propose GuardAgent, the first guardrail agent to protect target agents by dynamically checking whether their actions satisfy given safety guard requests. Specifically, GuardAgent first analyzes the safety guard requests to generate a task plan, and then maps this plan into guardrail code for execution. By performing the code execution, GuardAgent can deterministically follow the safety guard request and safeguard target agents. In both steps, an LLM is utilized as the reasoning component, supplemented by in-context demonstrations retrieved from a memory module storing experiences from previous tasks. In addition, we propose two novel benchmarks: EICU-AC benchmark to assess the access control for healthcare agents and Mind2Web-SC benchmark to evaluate the safety policies for web agents. We show that GuardAgent effectively moderates the violation actions for different types of agents on these two benchmarks with over 98% and 83% guardrail accuracies, respectively. Project page: https://guardagent.github.io/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑