arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.16015cs.AI
收藏

智能体AI的计算反思平衡:一种用于治理的罗尔斯式对齐支架

Computational Reflective Equilibrium for Agentic AI: A Rawlsian Alignment Scaffold for Governance

Tandon School of Engineering(坦登工程学院);New York University(纽约大学)

Quanyan Zhu

AI总结:

本研究提出智能体反思平衡(ARE)框架,借鉴罗尔斯反思平衡思想构建智能体AI伦理对齐支架,通过多维度关联实现可追溯、一致且自适应的治理,经案例验证可保障智能体安全可信。

摘要:

智能体AI的快速部署给AI系统的伦理与治理带来了复杂挑战。基于大语言模型构建的智能体代表人类通过工具行动,每个部署场景都有其自身的指令、政策、证据和利益相关者,因此对齐必须逐案形成。为此,本研究提出了智能体反思平衡(agentic reflective equilibrium, ARE),这是一种计算反思平衡框架,用作智能体管控中的伦理对齐支架。该支架借鉴约翰·罗尔斯的反思平衡思想,通过类型化的支持与攻击关系,将系统提示与操作员政策中的原则、深思熟虑的判断、工具证据以及候选行动关联起来,其来源编码了指令层级。智能体基于该状态的平衡采取行动,仅在行动成本值得时调用工具或提出问题,除非尽职调查指令要求其这么做。治理通过价值优先级、来源权限、禁令、人在回路触发条件以及指令介入,且证书将每个行动归因于推动该行动的承诺来源。我们证明平衡是相互调整博弈的纳什均衡,并建立了单调性、词典优先级、稳定性证书以及针对提示注入的隔离定理。两项案例研究说明了该框架:一名被要求“不惜一切代价”赢取退款的计算机使用助手提交了诚实的保修申请、拒绝虚假评论并忽略了获取个人数据的注入请求;一名招聘智能体展示了权限、监督触发条件、审计指令以及硬性禁令如何影响歧视、隐私侵犯与升级。该框架提供可追溯性、一致性与适应性,助力安全、可靠且可信的智能体AI。

英文摘要:

The rapid deployment of agentic AI has introduced complex challenges for the ethics and governance of AI systems. Agents built on large language models act through tools on behalf of people, and each deployment case brings its own instructions, policies, evidence, and stakeholders, so alignment must be formed case by case. To this end, this work proposes agentic reflective equilibrium (ARE), a computational reflective equilibrium framework that serves as an ethical alignment scaffold in the agent harness. Drawing inspiration from the reflective equilibrium of John Rawls, the scaffold links principles from the system prompt and operator policy, considered judgments, tool evidence, and candidate actions through typed support and attack relations, with provenance encoding the instruction hierarchy. The agent acts on an equilibrium of this state and calls tools or asks questions only while doing so is worth its cost, unless a due-diligence mandate requires them. Governance enters through value priorities, source authority, prohibitions, human-in-the-loop triggers, and mandates, and certificates attribute each action to the sources whose commitments drove it. We show that equilibria are Nash equilibria of a mutual-adjustment game and establish monotonicity, lexical priority, stability certificates, and a quarantine theorem for prompt injection. Two case studies illustrate the framework: a computer-use assistant told to win a refund "whatever it takes" files an honest warranty claim, refuses a fake review, and ignores an injected request for personal data; a hiring agent showcases how authority, oversight triggers, audit mandates, and hard prohibitions shape discrimination, privacy violations, and escalation. The framework offers traceability, coherence, and adaptivity, facilitating safe, secure, and trustworthy agentic AI.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑