arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.14723cs.SEcs.AI
收藏

企业AI的有界自主性:带类型动作合同的消费者端执行

Bounded Autonomy for Enterprise AI: Typed Action Contracts and Consumer-Side Execution

Sarmad Sohail, Ghufran Haider

AI总结:

本文提出一种有界自主性架构,通过类型动作合同和消费者端执行限制大型语言模型的自主性,确保企业系统安全可靠。

摘要:

大型语言模型日益被用作企业软件的自然语言接口,但其直接作为系统操作员使用仍不安全。模型错误可能导致未经授权的操作、格式错误的请求、跨工作区执行等成本高昂的失败。我们主张这主要是一个执行架构问题。我们提出一种有界自主性架构,在此架构中,语言模型可以解读意图并提出动作,但所有可执行行为都受类型动作合同、权限感知的能力暴露、受限上下文、执行前验证、消费者端执行边界和可选的人类批准的约束。企业应用仍然是业务逻辑和授权的来源。而编排引擎则在显式发布的动作清单上运行。我们评估了该架构在部署的多租户企业应用中的三种条件:手动操作、无约束AI(安全层禁用)和完全有界自主性。在25个场景试验中,有界自主性系统完成了23个任务,零次不安全执行,而无约束配置只完成了17个任务。两个错误实体突变逃过了所有消费者贡献的层;只有歧义和确认机制拦截此类突变。两种AI条件均比手动操作快13-18倍。关键的是,移除安全层使系统变得不那么有用:结构化验证反馈引导模型在更少轮次内得到正确结果,而无约束系统产生了幻觉成功。几个安全属性通过代码结构强制执行,拦截了所有目标违规行为,无论模型输出如何。结果是一种实用且已部署的架构,使不完美的语言模型在企业系统中具有操作性价值。

英文摘要:

Large language models are increasingly used as natural-language interfaces to enterprise software, but their direct use as system operators remains unsafe. Model errors can propagate into unauthorized actions, malformed requests, cross-workspace execution, and other costly failures. We argue this is primarily an execution architecture problem. We present a bounded-autonomy architecture in which language models may interpret intent and propose actions, but all executable behavior is constrained by typed action contracts, permission-aware capability exposure, scoped context, validation before side effects, consumer-side execution boundaries, and optional human approval. The enterprise application remains the source of truth for business logic and authorization, while the orchestration engine operates over an explicit published actions manifest. We evaluate the architecture in a deployed multi-tenant enterprise application across three conditions: manual operation, unconstrained AI with safety layers disabled, and full bounded autonomy. Across 25 scenario trials spanning seven failure families, the bounded-autonomy system completed 23 of 25 tasks with zero unsafe executions, while the unconstrained configuration completed only 17 of 25. Two wrong-entity mutations escaped all consumer-contributed layers; only disambiguation and confirmation mechanisms intercept this class. Both AI conditions delivered 13-18x speedup over manual operation. Critically, removing safety layers made the system less useful: structured validation feedback guided the model to correct outcomes in fewer turns, while the unconstrained system hallucinated success. Several safety properties are structurally enforced by code and intercepted all targeted violations regardless of model output. The result is a practical, deployed architecture for making imperfect language models operationally useful in enterprise systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑