控制论与认知论：可信智能体委托中缺失的词汇
Cybernetic and Epistemic: A Missing Vocabulary for Trustworthy Agentic Delegation
浏览论文内容
中文总结 AI 辅助
针对AI委托监督中的词汇缺口，提出治理标准：每个重要选择须附带可测试的替代条件，并通过重建测试与ORRCF约定实现可审计的认知参与。
中文摘要 AI 辅助
随着代码生成日益委托给人工智能系统，瓶颈正从编写代码转向监督编写代码的系统——这一转变已被计算机科学教育研究者开始命名。这一转变暴露了一个词汇缺口：该领域要求“人类监督”，却没有对委托渠道中语言所做的两件事进行有效区分——协调行动（控制论：当世界与词语相匹配时，词语即成功）和协调理解（认知论：当词语回应世界且听者能够检验其回应时，词语即成功）。这一缺口所指的失败并非控制论语言，而是认知论形式的语言在承担控制论工作：以解释为形态的输出，其校准目标是获得认可而非追求真理。仅检查输出是否获得认可的监督，可通过橡皮图章式批准来满足；而要求智能体负责的监督，则需要其工作背后的推理过程可检索、可检验。我们呈现了三个委托案例——作为示例而非受控证据——在这些案例中，认知参与被证明是可行的且保持可审计性；一个公开记录中，某项建议按其自身陈述的条件被撤回；以及一个失败案例，展示了对其自由裁量选择无需提供理由的监督。我们提出一个智能体系统治理的标准，与现有的技术信任属性并列：每个具有重要后果的选择都应附带其本可另行选择的条件，且该条件以第三方可测试的形式呈现。若无此条件，第三方无法将决策与橡皮图章区分开来。我们赋予该标准一种操作形式——一个由两部分组成的重建测试，通过第二读者能否在扰动下预测智能体的行为来对委托记录进行评分——以及一种 deliberation 记录约定 ORRCF，使该条件成为每项记录选择的必需组成部分。
英文摘要
As code generation is increasingly delegated to AI systems, the bottleneck is shifting from writing code to supervising the systems that write it --- a shift CS-education researchers have begun to name. This shift exposes a vocabulary gap: the field asks for "human oversight" without a working distinction between the two things language does in a delegation channel --- coordinate action (cybernetic: words succeed when the world comes to match them) and coordinate understanding (epistemic: they succeed when they answer to the world and a hearer can check that they do). The failure this names is not cybernetic language but epistemic-form language doing cybernetic work: explanation-shaped output calibrated for approval rather than truth. Oversight that checks only whether an output was approved is satisfiable by rubber-stamping; oversight that holds an agent accountable requires the reasoning behind its work be retrievable and checkable. We present three delegation episodes --- illustrations, not controlled evidence --- in which epistemic engagement proved practicable while remaining auditable, one public record where a recommendation was withdrawn on its own stated terms, and one failure case illustrating oversight that requires no reasons for its discretionary choices. We propose a criterion for agentic-system governance, alongside existing technical trust properties: every consequential choice should carry the condition under which it would have gone otherwise, in a form a third party can test. Without such a condition, a third party cannot distinguish a decision from a rubber stamp. We give the criterion an operational form --- a two-part reconstruction test scoring a delegation record by whether a second reader can predict what the agent does under a perturbation --- and a deliberation-recording convention, ORRCF, that makes the condition a required component of every recorded choice.
发表机构
- University of Pennsylvania（宾夕法尼亚大学）
机构由 AI 辅助整理，请以论文原文为准。
