效用保持的可校正性
Corrigibility with Utility Preservation
摘要：
可纠正性是人工智能代理的安全属性。可纠正代理不会抵制授权方更改代理首次启动时编码的目标和约束的尝试。本文展示了如何构建一个安全层，增加了可纠正性任意先进的效用最大化代理，包括未来可能的代理与人工通用智能（AGI）。这一层抵消了高级代理人抵制这种改变的紧急激励。一个详细的代理模型，可以保留他们的效用函数的原因，并用于证明，corrigibility层的工作原理，如预期在一个大的非敌对的宇宙。可矫正的代理人有一个紧急的激励，以保护他们的可矫正层的关键元素。然而，敌对的宇宙可能包含强大到足以破坏安全功能的力量。一些开放的问题有关的优雅退化时，一个代理被成功攻击。本文的结果是通过同时开发AGI代理模拟器，代理模型和证明得到的。该模拟器在开源许可证下可用。本文包含的模拟结果，详细说明了可纠正AGI代理的安全相关属性。
英文摘要：
Corrigibility is a safety property for artificially intelligent agents. A corrigible agent will not resist attempts by authorized parties to alter the goals and constraints that were encoded in the agent when it was first started. This paper shows how to construct a safety layer that adds corrigibility to arbitrarily advanced utility maximizing agents, including possible future agents with Artificial General Intelligence (AGI). The layer counter-acts the emergent incentive of advanced agents to resist such alteration. A detailed model for agents which can reason about preserving their utility function is developed, and used to prove that the corrigibility layer works as intended in a large set of non-hostile universes. The corrigible agents have an emergent incentive to protect key elements of their corrigibility layer. However, hostile universes may contain forces strong enough to break safety features. Some open problems related to graceful degradation when an agent is successfully attacked are identified. The results in this paper were obtained by concurrently developing an AGI agent simulator, an agent model, and proofs. The simulator is available under an open source license. The paper contains simulation results which illustrate the safety related properties of corrigible AGI agents in detail.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
