可控性陷阱:军事AI代理的治理框架
The Controllability Trap: A Governance Framework for Military AI Agents
AI总结:
本文提出AMAGF框架,通过预防、检测和纠正三个支柱,解决军事AI代理中的控制失效问题,通过控制质量评分实现持续控制管理。
摘要:
具备目标解释、世界建模、规划、工具使用、长周期运行和自主协调能力的代理AI系统引入了现有安全框架无法解决的控制失效问题。我们识别出六种与这些能力相关的代理治理失效,并展示了它们如何在军事环境中削弱有意义的人类控制。我们提出了代理军事AI治理框架(AMAGF),一种可衡量的架构,围绕三个支柱:预防性治理(减少失效可能性)、检测性治理(实时检测控制退化)和纠正性治理(恢复或安全降级操作)。其核心机制,控制质量评分(CQS),是一种复合实时指标,用于量化人类控制并允许在控制减弱时采取渐进响应。对于每种失效类型,我们定义了具体的机制,分配了五个机构主体的责任,并正式化了评估指标。一个工作操作场景展示了实施方式,我们还将该框架置于已建立的代理安全文献中。我们认为,治理必须从二元的控制概念转向连续模型,在整个操作生命周期中积极测量和管理控制质量。
英文摘要:
Agentic AI systems - capable of goal interpretation, world modeling, planning, tool use, long-horizon operation, and autonomous coordination - introduce distinct control failures not addressed by existing safety frameworks. We identify six agentic governance failures tied to these capabilities and show how they erode meaningful human control in military settings. We propose the Agentic Military AI Governance Framework (AMAGF), a measurable architecture structured around three pillars: Preventive Governance (reducing failure likelihood), Detective Governance (real-time detection of control degradation), and Corrective Governance (restoring or safely degrading operations). Its core mechanism, the Control Quality Score (CQS), is a composite real-time metric quantifying human control and enabling graduated responses as control weakens. For each failure type, we define concrete mechanisms, assign responsibilities across five institutional actors, and formalize evaluation metrics. A worked operational scenario illustrates implementation, and we situate the framework within established agent safety literature. We argue that governance must move from a binary conception of control to a continuous model in which control quality is actively measured and managed throughout the operational lifecycle.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。