arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.11749cs.AI
收藏

AIR:通过事件响应提升智能体安全性

AIR: Improving Agent Safety through Incident Response

Tianjin University(天津大学);Singapore Management University(新加坡国立管理学院)

Zibo Xiao, Jun Sun, Junjie Chen

AI总结:

提出首个面向LLM智能体系统的事件响应框架AIR,通过语义检测、自动遏制恢复和规则合成,在三个典型智能体上实现超90%的检测、修复和根除成功率。

摘要:

大型语言模型(LLM)智能体正越来越多地部署在各类自主应用中。然而,当前LLM智能体的安全机制几乎完全专注于提前预防故障,在事件不可避免地发生后,缺乏响应、遏制或恢复的能力。在这项工作中,我们引入了AIR,这是首个面向LLM智能体系统的事件响应框架。AIR定义了一种领域特定语言,用于自主管理LLM智能体系统中的事件响应生命周期,并将其集成到智能体的执行循环中,以(1)通过基于当前环境状态和近期上下文的语义检查检测事件,(2)引导智能体通过其工具执行遏制和恢复操作,以及(3)在根除阶段合成防护栏规则,以阻止未来执行中类似事件的发生。我们在三种代表性智能体类型上评估了AIR。结果表明,AIR的检测、修复和根除成功率均超过90%。大量实验进一步证实了AIR关键设计组件的必要性,显示了AIR的及时性和适度开销,并证明了LLM生成的规则可以跨领域接近开发者编写规则的有效性。这些结果表明,事件响应作为提升智能体安全性的首要机制既可行又至关重要。

英文摘要:

Large Language Model (LLM) agents are increasingly deployed in practice across a wide range of autonomous applications. Yet current safety mechanisms for LLM agents focus almost exclusively on preventing failures in advance, providing limited capabilities for responding to, containing, or recovering from incidents after they inevitably arise. In this work, we introduce AIR, the first incident response framework for LLM agent systems. AIR defines a domain-specific language for managing the incident response lifecycle autonomously in LLM agent systems, and integrates it into the agent's execution loop to (1) detect incidents via semantic checks grounded in the current environment state and recent context, (2) guide the agent to execute containment and recovery actions via its tools, and (3) synthesize guardrail rules during eradication to block similar incidents in future executions. We evaluate AIR on three representative agent types. Results show that AIR achieves detection, remediation, and eradication success rates all exceeding 90%. Extensive experiments further confirm the necessity of AIR's key design components, show the timeliness and moderate overhead of AIR, and demonstrate that LLM-generated rules can approach the effectiveness of developer-authored rules across domains. These results show that incident response is both feasible and essential as a first-class mechanism for improving agent safety.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑