arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.09947cs.CR
收藏

可信的代理AI需要确定性的架构边界

Trustworthy Agentic AI Requires Deterministic Architectural Boundaries

Los Alamos National Laboratory(洛斯阿拉莫斯国家实验室)

Manish Bhattarai, Minh Vu

AI总结:

本文提出三位一体防御架构,通过确定性的架构执行解决代理AI的安全问题,强调架构调解对可信AI应用的必要性。

摘要:

当前的代理AI架构在安全性和认识论要求上与高风险的科学工作流程根本不可兼容。问题不在于对齐不足或防护不足,而是架构层面的:自回归语言模型对所有标记进行统一处理,使得通过训练单独实现确定性的命令-数据分离是不可能的。我们主张,确定性的架构执行,而不是概率性的学习行为,是可信AI辅助科学的必要条件。我们介绍了三位一体防御架构,通过三个机制实施安全:通过有限动作算术与参考监控执行动作治理,通过强制访问标签控制信息流以防止跨范围泄漏,以及通过特权分离隔离感知与执行。我们显示,在不可伪造的溯源和确定性调解不存在的情况下,"致命三联症"(不可信输入、特权数据访问、外部行动能力)使授权安全变成了漏洞发现问题:基于训练的防御可能减少实际攻击率,但无法提供确定性的保证。机器学习社区必须认识到,对齐不足以满足授权安全,且在代理AI能够安全地应用于关键科学领域之前,需要架构调解。

英文摘要:

Current agentic AI architectures are fundamentally incompatible with the security and epistemological requirements of high-stakes scientific workflows. The problem is not inadequate alignment or insufficient guardrails, it is architectural: autoregressive language models process all tokens uniformly, making deterministic command--data separation unattainable through training alone. We argue that deterministic, architectural enforcement, not probabilistic learned behavior, is a necessary condition for trustworthy AI-assisted science. We introduce the Trinity Defense Architecture, which enforces security through three mechanisms: action governance via a finite action calculus with reference-monitor enforcement, information-flow control via mandatory access labels preventing cross-scope leakage, and privilege separation isolating perception from execution. We show that without unforgeable provenance and deterministic mediation, the ``Lethal Trifecta'' (untrusted inputs, privileged data access, external action capability) turns authorization security into an exploit-discovery problem: training-based defenses may reduce empirical attack rates but cannot provide deterministic guarantees. The ML community must recognize that alignment is insufficient for authorization security, and that architectural mediation is required before agentic AI can be safely deployed in consequential scientific domains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑