Large Language Model (LLM) agents are increasingly deployed in practice across a wide range of autonomous applications. Yet current safety mechanisms for LLM agents focus almost exclusively on preventing failures in advance, providing limited capabilities for responding to, containing, or recovering from incidents after they inevitably arise. In this work, we introduce AIR, the first incident response framework for LLM agent systems. AIR defines a domain-specific language for managing the incident response lifecycle autonomously in LLM agent systems, and integrates it into the agent's execution loop to (1) detect incidents via semantic checks grounded in the current environment state and recent context, (2) guide the agent to execute containment and recovery actions via its tools, and (3) synthesize guardrail rules during eradication to block similar incidents in future executions. We evaluate AIR on three representative agent types. Results show that AIR achieves detection, remediation, and eradication success rates all exceeding 90%. Extensive experiments further confirm the necessity of AIR's key design components, show the timeliness and moderate overhead of AIR, and demonstrate that LLM-generated rules can approach the effectiveness of developer-authored rules across domains. These results show that incident response is both feasible and essential as a first-class mechanism for improving agent safety.
Towards Automated Air Traffic Safety Assessment Around Non-Towered Airports Using Large Language Models
Research Intern, Department of Mechanical and Aerospace Engineering, George Washington University(乔治华盛顿大学机械与航空航天工程系研究实习生);Ph.D. Student, Department of Mechanical and Aerospace Engineering, George Washington University(乔治华盛顿大学机械与航空航天工程系博士生);Undergraduate Student, Aerospace Program, University of California, Berkeley(加州大学伯克利分校航空航天项目本科生);Full Professor, Department of Electrical Engineering and Computer Science, University of California, Berkeley(加州大学伯克利分校电气工程与计算机科学系教授);Ph.D. Student, Department of Computer Science, George Washington University(乔治华盛顿大学计算机科学系博士生);Associate Professor, Department of Mechanical and Aerospace Engineering, George Washington University(乔治华盛顿大学机械与航空航天工程系副教授)
Automated Triaging and Transfer Learning of Incident Learning Safety Reports Using Large Language Representational Models
University of Washington School of Medicine(华盛顿大学医学院);University of Washington, Department of Biomedical Informatics and Medical Education(华盛顿大学生物医学信息学与医学教育系);International Atomic Energy Agency(国际原子能机构);University of Washington School of Medicine, Department of Radiation Oncology(华盛顿大学医学院放射肿瘤学系)
AIR-BENCH Live: An Evolving Safety Benchmark for Foundation Models
Virtue AI(美德人工智能公司);Carnegie Mellon University(卡内基梅隆大学);Northeastern University(东北大学);University of Chicago(芝加哥大学);University of Illinois, Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policies
Virginia Tech(弗吉尼亚理工大学);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);Lapis Labs(拉皮斯实验室);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of California, Berkeley(加利福尼亚大学伯克利分校);Stanford University(斯坦福大学);Harvard University(哈佛大学);Northeastern University(东北大学);University of Chicago(芝加哥大学)
Semantic Causal-Factor Inference from Aviation Incident Narratives Using A Variational Autoencoder with Cosine-Similarity-Based Reconstruction
SDAIA-KFUPM Joint Research Center for Artificial Intelligence, King Fahd University of Petroleum and Minerals (KFUPM)(SDAIA-KFUPM人工智能联合研究中心,法赫德国王石油与矿产大学);University of New South Wales(新南威尔士大学)
AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies
Virginia Tech(弗吉尼亚理工大学);Stanford University(斯坦福大学);Harvard University(哈佛大学);Lapis Labs(拉皮斯实验室);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);Northeastern University(东北大学);University of California, Berkeley(加利福尼亚大学伯克利分校);University of Chicago(芝加哥大学)