arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.24686cs.AI
收藏

治理你无法观察的事物:面向自主AI代理的自适应运行时治理

Governing What You Cannot Observe: Adaptive Runtime Governance for Autonomous AI Agents

Department of Computing, University of Turku(图尔库大学计算机系)

German Marin, Jatin Chaudhary

AI总结:

本文提出信息有效性原理,通过估计未观察风险上界并设定安全余量,实现自主AI代理的自适应运行时治理,结合Aubin viability理论构建Agent有效性框架,引入风险门机制实现预测性治理。

摘要:

自主AI代理在行为漂移、对手适应和决策模式变化时仍可能变得不安全,即使没有代码更改。我们提出信息有效性原理:治理代理等同于估计未观察风险上界$\hat{B}(x) = U(x) + SB(x) + RG(x)$,并仅在容量$S(x)$超过$\hat{B}(x)$时允许行动。基于Aubin的可行性理论,Agent有效性框架建立了三个属性——监控(P1)、预见(P2)和单调限制(P3)——作为文档化故障模式的必要且充分条件。RiskGate通过专用统计估计器(KL散度、段与其余$z$检验、序列模式匹配)、故障安全单调管道和闭环Autopilot(作为Aubin调节映射的实例,杀开关作为最后手段)实例化该框架;标量可行性指数$VI(t) \in [-1,+1]$和一阶$t^*$预测将治理从反应式转为预测式。贡献包括理论框架、参考实现和对已发布代理故障分类的分析覆盖;定量实证评估作为后续工作。

英文摘要:

Autonomous AI agents can remain fully authorized and still become unsafe as behavior drifts, adversaries adapt, and decision patterns shift without any code change. We propose the \textbf{Informational Viability Principle}: governing an agent reduces to estimating a bound on unobserved risk $\hat{B}(x) = U(x) + SB(x) + RG(x)$ and allowing an action only when its capacity $S(x)$ exceeds $\hat{B}(x)$ by a safety margin. The \textbf{Agent Viability Framework}, grounded in Aubin's viability theory, establishes three properties -- monitoring (P1), anticipation (P2), and monotonic restriction (P3) -- as individually necessary and collectively sufficient for documented failure modes. \textbf{RiskGate} instantiates the framework with dedicated statistical estimators (KL divergence, segment-vs-rest $z$-tests, sequential pattern matching), a fail-secure monotonic pipeline, and a closed-loop Autopilot formalised as an instance of Aubin's regulation map with kill-switch-as-last-resort; a scalar Viability Index $VI(t) \in [-1,+1]$ with first-order $t^*$ prediction transforms governance from reactive to predictive. Contributions are the theoretical framework, the reference implementation, and analytical coverage against published agent-failure taxonomies; quantitative empirical evaluation is scoped as follow-up work.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑