arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

当智能体成为内核:通向AI原生操作系统之路上的安全系统化研究

When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systems

Li Zhang, Yang Sun, Jie Shi

arXiv 2609.23700首次发表:更新:

发表机构

Huawei(华为)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

本文系统化研究AI原生操作系统的安全,提出基于溯源与语义的中介区分,指出核心中介缺口,并探讨模型作为仲裁核心的未来架构。

AI 中文摘要

大语言模型智能体现在已成为拥有特权的执行主体,能够采取具有重大影响的行动:编辑代码仓库、操作收件箱、完成购买。它们的权限级别堪比内核,但缺乏经典系统安全所要求的条件:在每次访问时介入的可信中介。操作系统供应商现在正围绕这一事实上的智能体内核重建平台,将完全中介作为设计问题加以继承。我们围绕一个核心区分来系统化此类系统的安全性:经由溯源进行中介的跨越允许确定性检查,而基于内容语义的跨越则不允许。一个信任边界分类法定位了必须进行中介的位置,并将核心中介缺口隔离在两类语义判断上:在不可信输入中区分数据与指令,以及区分授权行为与非授权行为。我们认为,只要输入和行为未事先限制在枚举集合内,这一缺口就会留下不可约的未检测攻击残余。同样的区分使攻击成功统计数据具有可操作性,将每个数字置于从部署债务(一个健全的确定性中介未被使用)到结构性缺口(不存在此类已知中介)的谱系上。我们系统化了跨运行时监控、架构分离和授权的防御措施,并表明当前评估往往通过评估有效性失败而高估了已部署的安全性。最后,我们将这一分析推进到超越事实上的内核,进入一种模型本身成为仲裁核心的架构,并推导出安全优先的AI原生操作系统的设计约束、开放挑战和研究议程。

英文摘要

Large language model agents are now privileged principals that take consequential actions: editing code repositories, operating inboxes, completing purchases. Their authority is kernel-grade, but it comes without what classical systems security requires: a trusted mediator interposed on every access. Operating-system vendors are now rebuilding the platform around this de-facto agent kernel, inheriting complete mediation as a design problem. We systematize the security of such systems around a single distinction: a crossing mediated over provenance admits a deterministic check, while one over content semantics does not. A trust-boundary taxonomy locates where mediation must occur and isolates the central mediation gap at two kinds of semantic judgment: distinguishing data from instruction in untrusted input, and an authorized action from an unauthorized one. We argue that this gap leaves an irreducible residual of undetected attacks wherever inputs and actions are not restricted in advance to an enumerated set. The same distinction makes attack-success statistics actionable, placing each number on a spectrum from deployment debt (a sound deterministic mediator left unused) to a structural gap (no such mediator known). We systematize defenses across runtime monitoring, architectural separation, and authorization, and show that current evaluations tend to overstate deployed security through evaluation-validity failures. Finally, we carry that analysis forward beyond the de-facto kernel, to an architecture in which the model itself becomes the arbitration core, and derive the design constraints, open challenges, and research agenda for a security-first AI-native OS.

Comments32 pages, 5 figures, 6 tables

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑