arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1605.03142cs.AI
收藏

理性Agent的策略自修正与效用函数

Self-Modification of Policy and Utility Function in Rational Agents

Australian National University(澳大利亚国立大学)

Tom Everitt, Daniel Filan, Mayank Daswani, Marcus Hutter

摘要:

任何与环境交互的智能体,如果它有多功能的执行器(比如手臂和手指),原则上都有自我修改的能力,比如修改自己的源代码。随着我们继续创造越来越多的智能体,他们学习这种能力的机会也会增加。问题是:他们愿意使用它吗?例如,高度智能的系统可能会找到方法将其目标改变为更容易实现的目标,从而“逃脱”其设计者的控制。Omohundro(2008)在一篇重要的论文中指出,目标保持是任何智能系统的基本驱动力,因为如果未来版本的智能体朝着同一目标努力,那么目标更有可能实现。在本文中,我们将这一论点形式化为一般的强化学习,并探讨它失败的情况。我们的结论是,自我修改的可能性是无害的,当且仅当价值函数的代理预期的后果自我修改,并使用当前的效用函数时,评估未来。

英文摘要:

Any agent that is part of the environment it interacts with and has versatile actuators (such as arms and fingers), will in principle have the ability to self-modify -- for example by changing its own source code. As we continue to create more and more intelligent agents, chances increase that they will learn about this ability. The question is: will they want to use it? For example, highly intelligent systems may find ways to change their goals to something more easily achievable, thereby `escaping' the control of their designers. In an important paper, Omohundro (2008) argued that goal preservation is a fundamental drive of any intelligent system, since a goal is more likely to be achieved if future versions of the agent strive towards the same goal. In this paper, we formalise this argument in general reinforcement learning, and explore situations where it fails. Our conclusion is that the self-modification possibility is harmless if and only if the value function of the agent anticipates the consequences of self-modifications and use the current utility function when evaluating the future.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑