arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2602.01942cs.CRcs.AI

面向代理AI安全的人类社会启发方法:4C框架

Human Society-Inspired Approaches to Agentic AI Security: The 4C Framework

University of New South Wales, Sydney(新南威尔士大学悉尼分校)

Alsharif Abuadbba, Nazatul Sultan, Surya Nepal, Sanjay Jha

AI总结:

本文提出4C框架,通过人类社会治理理念,为代理AI安全提供多维度保护,强调行为完整性和意图,构建可信可控的AI系统。

摘要:

AI正从封闭、可预测环境中的领域特定自主性,向由大语言模型驱动的代理AI系统发展,这些系统能够在开放、跨组织环境中进行计划和行动。因此,网络安全风险图景正在发生根本性的变化。代理AI系统能够计划、行动、协作并持续存在,作为复杂社会技术生态系统中的参与者,而非孤立的软件组件。尽管近期的工作已加强了对模型和流程层面漏洞(如提示注入、数据污染和工具滥用)的防御,但这些以系统为中心的方法可能无法捕捉到由自主性、交互和涌现行为带来的风险。本文介绍了一个受社会治理启发的多代理AI安全4C框架。该框架将代理风险划分为四个相互依赖的维度:核心(系统、基础设施和环境完整性)、连接(通信、协调和信任)、认知(信念、目标和推理完整性)以及合规性(伦理、法律和制度治理)。通过将AI安全从以系统为中心的保护窄聚焦,转向更广泛的保护行为完整性和意图,该框架补充了现有的AI安全策略,并为构建可信、可控且与人类价值观一致的代理AI系统提供了原则性基础。

英文摘要:

AI is moving from domain-specific autonomy in closed, predictable settings to large-language-model-driven agents that plan and act in open, cross-organizational environments. As a result, the cybersecurity risk landscape is changing in fundamental ways. Agentic AI systems can plan, act, collaborate, and persist over time, functioning as participants in complex socio-technical ecosystems rather than as isolated software components. Although recent work has strengthened defenses against model and pipeline level vulnerabilities such as prompt injection, data poisoning, and tool misuse, these system centric approaches may fail to capture risks that arise from autonomy, interaction, and emergent behavior. This article introduces the 4C Framework for multi-agent AI security, inspired by societal governance. It organizes agentic risks across four interdependent dimensions: Core (system, infrastructure, and environmental integrity), Connection (communication, coordination, and trust), Cognition (belief, goal, and reasoning integrity), and Compliance (ethical, legal, and institutional governance). By shifting AI security from a narrow focus on system-centric protection to the broader preservation of behavioral integrity and intent, the framework complements existing AI security strategies and offers a principled foundation for building agentic AI systems that are trustworthy, governable, and aligned with human values.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑