arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.22211cs.AI
收藏

能力越大责任越大:引入代理风险与能力框架以治理代理AI系统

With Great Capabilities Come Great Responsibilities: Introducing the Agentic Risk & Capability Framework for Governing Agentic AI Systems

GovTech Singapore(新加坡政府科技局);Singapore University of Technology and Design(新加坡科技与设计大学)

Shaun Khoo, Jessica Foo, Roy Ka-Wei Lee

AI总结:

本文提出代理风险与能力框架,用于治理代理AI系统,通过能力中心视角和风险来源分析,帮助组织有效管理AI系统风险。

摘要:

代理AI系统由于其自主行动的能力,如代码执行、互联网交互和文件修改等任务,带来了显著的机会和新的风险。这给有效的组织治理带来了巨大挑战,特别是在全面识别、评估和缓解多样且不断演变的风险方面。为此,我们引入了代理风险与能力(ARC)框架,这是一个技术治理框架,旨在帮助组织识别、评估和缓解来自代理AI系统的风险。该框架的核心贡献是:(1)它开发了一种新的以能力为中心的视角来分析广泛范围的代理AI系统;(2)它提炼出三个主要的风险来源,即组件、设计和能力;(3)它建立了每个风险来源、具体的实质性风险和相应的技术控制之间的明确联系;(4)它提供了一种结构化和实用的方法,帮助组织实施该框架。该框架为组织提供了稳健且可调整的方法,以应对代理AI的复杂性,使组织能够快速有效地进行创新,同时确保代理AI系统的安全、安全和负责任的部署。我们的框架已开源,详见https://govtech-responsibleai.github.io/agentic-risk-capability-framework/。

英文摘要:

Agentic AI systems present both significant opportunities and novel risks due to their capacity for autonomous action, encompassing tasks such as code execution, internet interaction, and file modification. This poses considerable challenges for effective organizational governance, particularly in comprehensively identifying, assessing, and mitigating diverse and evolving risks. To tackle this, we introduce the Agentic Risk \& Capability (ARC) Framework, a technical governance framework designed to help organizations identify, assess, and mitigate risks arising from agentic AI systems. The framework's core contributions are: (1) it develops a novel capability-centric perspective to analyze a wide range of agentic AI systems; (2) it distills three primary sources of risk intrinsic to agentic AI systems - components, design, and capabilities; (3) it establishes a clear nexus between each risk source, specific materialized risks, and corresponding technical controls; and (4) it provides a structured and practical approach to help organizations implement the framework. This framework provides a robust and adaptable methodology for organizations to navigate the complexities of agentic AI, enabling rapid and effective innovation while ensuring the safe, secure, and responsible deployment of agentic AI systems. Our framework is open-sourced \href{https://govtech-responsibleai.github.io/agentic-risk-capability-framework/}{here}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑