arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.09045cs.AIcs.CRcs.SE
收藏

包含性验证:与对齐无关的AI安全保证

Containment Verification: AI Safety Guarantees Independent of Alignment

AI Innovation Institute, Stony Brook University(石溪大学人工智能创新研究所)

Royce Moon, Lav R. Varshney

AI总结:

本文提出包含性验证,通过代理框架本身提供安全保证。通过前向模拟细化和Dafny机制化,证明了在模型类型动作边界内对所有AI输出的边界策略强制性,首次实现了代理框架的演绎形式验证。

摘要:

代理框架是AI代理在世界中行动的软件层。现有安全方法干预模型,因此依赖于不可验证的学习行为属性。我们引入包含性验证,将安全保证定位在代理框架本身。在havoc oracle语义下,AI被建模为一个无约束的oracle,覆盖整个类型化动作空间,验证的包含层必须对每个可能的AI输出强制执行边界策略。对于可强制执行的属性,通过前向模拟细化证明了普遍保证,并在Dafny中机制化。通过验证PocketFlow,一个最小化的代理LLM框架,并使用代理合成管道生成规范、操作模型和细化证明,以对抗tautological规范。据我们所知,这是首次对代理框架的演绎形式验证,其保证在模型能力范围内与类型化动作边界无关。

英文摘要:

Agentic frameworks are the software layer through which AI agents act in the world. Existing safety methods intervene on the model and therefore remain conditional on unverifiable properties of learned behavior. We introduce containment verification, which locates safety guarantees in the agentic framework itself. Under havoc oracle semantics, the AI is modeled as an unconstrained oracle over the framework's typed action space, and the verified containment layer must enforce the boundary policy for every typed action value the AI can emit. For boundary-enforceable properties, expressed over modeled boundary events, action arguments, and state, we prove a universal guarantee by forward-simulation refinement and mechanize it in Dafny. We instantiate the paradigm by verifying PocketFlow, a minimalist agentic LLM framework, and use an agentic synthesis pipeline to generate the specification, operational model, and refinement proof under an information barrier against tautological specifications. To our knowledge, this is the first deductive formal verification of an agentic framework. The guarantee is independent of alignment because it quantifies over the framework's typed action boundary rather than over model behavior.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑