arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1912.01683cs.AI
收藏

最优政策趋向于权力寻租

Optimal Policies Tend to Seek Power

Oregon State University(俄勒冈州立大学);Mississippi State University(密西西比州立大学);UC Berkeley(加州大学伯克利分校)

Alexander Matt Turner, Logan Smith, Rohin Shah, Andrew Critch, Prasad Tadepalli

摘要:

一些研究人员推测,智能强化学习(RL)代理将被激励去寻求资源和权力来追求他们的目标。其他研究人员指出,强化学习智能体不需要像人类一样具有寻求权力的本能。为了澄清这一讨论,我们开发的最优政策的统计趋势的第一个正式的理论。在马尔可夫决策过程的背景下,我们证明了一定的环境对称性是足够的最优政策,往往寻求权力的环境。这些对称性存在于许多环境中,在这些环境中,代理可以被关闭或销毁。我们证明,在这些环境中,大多数奖励功能,使其最佳的寻求权力,保持一系列的选项,最大化平均奖励时,通过导航到更大的潜在终端状态。

英文摘要:

Some researchers speculate that intelligent reinforcement learning (RL) agents would be incentivized to seek resources and power in pursuit of their objectives. Other researchers point out that RL agents need not have human-like power-seeking instincts. To clarify this discussion, we develop the first formal theory of the statistical tendencies of optimal policies. In the context of Markov decision processes, we prove that certain environmental symmetries are sufficient for optimal policies to tend to seek power over the environment. These symmetries exist in many environments in which the agent can be shut down or destroyed. We prove that in these environments, most reward functions make it optimal to seek power by keeping a range of options available and, when maximizing average reward, by navigating towards larger sets of potential terminal states.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑