AI安全的瑞士奶酪模型：基于基础模型的智能体多层护栏分类法与参考架构
Swiss Cheese Model for AI Safety: A Taxonomy and Reference Architecture for Multi-Layered Guardrails of Foundation Model Based Agents
AI总结：
本文针对基于基础模型的智能体运行时安全问题，提出了运行时护栏的综合分类法，并受瑞士奶酪模型启发设计了包含质量属性、流水线和产物三个维度的多层护栏参考架构，为构建AI安全设计提供指导。
摘要：
基于基础模型（FM）的智能体正在彻底改变各个领域的应用开发。然而，其快速增长的自主性和能力引发了对AI安全的严重担忧。研究人员正在探索更好的方法来设计护栏，以确保基于FM的智能体的运行时行为保持在特定边界内。尽管如此，由于智能体的自主和非确定性行为，设计有效的运行时护栏极具挑战性。运行时多个流水线阶段和智能体产物（如目标、计划和工具）的参与进一步使这些问题复杂化。在运行时解决这些挑战需要在智能体架构的各个层面上有效运作的多层护栏。因此，本文基于系统性文献综述的结果，提出了基于FM的智能体运行时护栏的综合分类法，以识别护栏的关键质量属性和设计维度。受瑞士奶酪模型启发，我们还提出了一种用于设计基于FM的智能体多层运行时护栏的参考架构，该架构包含三个维度：质量属性、流水线和产物。所提出的分类法和参考架构为研究人员和从业者从软件架构角度构建“设计即安全”的AI提供了具体且稳健的指导。
英文摘要：
Foundation Model (FM)-based agents are revolutionizing application development across various domains. However, their rapidly growing capabilities and autonomy have raised significant concerns about AI safety. Researchers are exploring better ways to design guardrails to ensure that the runtime behavior of FM-based agents remains within specific boundaries. Nevertheless, designing effective runtime guardrails is challenging due to the agents' autonomous and non-deterministic behavior. The involvement of multiple pipeline stages and agent artifacts, such as goals, plans, tools, at runtime further complicates these issues. Addressing these challenges at runtime requires multi-layered guardrails that operate effectively at various levels of the agent architecture. Therefore, in this paper, based on the results of a systematic literature review, we present a comprehensive taxonomy of runtime guardrails for FM-based agents to identify the key quality attributes for guardrails and design dimensions. Inspired by the Swiss Cheese Model, we also propose a reference architecture for designing multi-layered runtime guardrails for FM-based agents, which includes three dimensions: quality attributes, pipelines, and artifacts. The proposed taxonomy and reference architecture provide concrete and robust guidance for researchers and practitioners to build AI-safety-by-design from a software architecture perspective.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
