参数可重定向决策者倾向于寻求权力
Parametrically Retargetable Decision-Makers Tend To Seek Power
摘要:
如果有能力的人工智能智能体通常被激励去寻求权力,以服务于我们为他们指定的目标,那么这些系统除了带来巨大的好处外,还将带来巨大的风险。在完全可观察的环境中,大多数奖励函数都有一个最优策略,它通过保持选项开放和生存来寻求权力。然而,现实世界既不是完全可观察的,也不需要训练的代理人甚至近似于奖励最优。我们考虑了一系列人工智能决策模型,从最优到随机,再到通过学习和与环境交互而做出的选择。我们发现,许多决策功能是可重定向的,而重定向性足以导致权力寻求倾向。我们的功能标准是简单和广泛的。我们表明,一系列的质量不同的决策程序激励代理人寻求权力。我们证明了我们的研究结果的灵活性,通过推理学习的政策激励蒙特祖马的复仇。这些结果暗示了一个安全风险:最终,可重定向的训练程序可能会训练现实世界中寻求超越人类的力量的代理人。
英文摘要:
If capable AI agents are generally incentivized to seek power in service of the objectives we specify for them, then these systems will pose enormous risks, in addition to enormous benefits. In fully observable environments, most reward functions have an optimal policy which seeks power by keeping options open and staying alive. However, the real world is neither fully observable, nor must trained agents be even approximately reward-optimal. We consider a range of models of AI decision-making, from optimal, to random, to choices informed by learning and interacting with an environment. We discover that many decision-making functions are retargetable, and that retargetability is sufficient to cause power-seeking tendencies. Our functional criterion is simple and broad. We show that a range of qualitatively dissimilar decision-making procedures incentivize agents to seek power. We demonstrate the flexibility of our results by reasoning about learned policy incentives in Montezuma's Revenge. These results suggest a safety risk: Eventually, retargetable training procedures may train real-world agents which seek power over humans.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。