arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1905.04640cs.AI
收藏

Mega-Reward:在没有额外奖励的情况下实现人类水平的游戏

Mega-Reward: Achieving Human-Level Play without Extrinsic Rewards

University of Oxford(牛津大学);Beihang University(北京航空航天大学);Hebei University of Technology(河北工业大学)

Yuhang Song, Jianyi Wang, Thomas Lukasiewicz, Zhenghua Xu, Shangtong Zhang, Andrzej Wojcicki, Mai Xu

摘要:

内在奖励被引入来模拟人类智力的工作方式;它们通常通过内在动机的游戏来评估,即,玩没有外在奖励但有外在奖励评估的游戏。然而,没有一个现有的内在奖励方法可以实现人类水平的性能下,这个非常具有挑战性的设置内在动机的发挥。在这项工作中,我们提出了一种新的自大驱动的内在奖励(称为mega-reward),据我们所知,这是第一种在内在动机游戏中实现人类水平表现的方法。直觉上,超级奖励来自于观察到婴儿的智力发展,当他们试图获得更多的控制实体在一个环境中;因此,超级奖励的目的是最大限度地提高控制能力的代理人对给定的实体在一个给定的环境。为了形式化大奖励,提出了一个关系转换模型,以弥合直接和潜在控制之间的差距。实验研究表明,超级奖励(i)可以大大优于所有最先进的内在奖励方法,(ii)通常达到与Ex-PPO和专业人类水平分数相同的性能水平,(iii)当与外在奖励结合时,也具有优越的性能。

英文摘要:

Intrinsic rewards were introduced to simulate how human intelligence works; they are usually evaluated by intrinsically-motivated play, i.e., playing games without extrinsic rewards but evaluated with extrinsic rewards. However, none of the existing intrinsic reward approaches can achieve human-level performance under this very challenging setting of intrinsically-motivated play. In this work, we propose a novel megalomania-driven intrinsic reward (called mega-reward), which, to our knowledge, is the first approach that achieves human-level performance in intrinsically-motivated play. Intuitively, mega-reward comes from the observation that infants' intelligence develops when they try to gain more control on entities in an environment; therefore, mega-reward aims to maximize the control capabilities of agents on given entities in a given environment. To formalize mega-reward, a relational transition model is proposed to bridge the gaps between direct and latent control. Experimental studies show that mega-reward (i) can greatly outperform all state-of-the-art intrinsic reward approaches, (ii) generally achieves the same level of performance as Ex-PPO and professional human-level scores, and (iii) has also a superior performance when it is incorporated with extrinsic rewards.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑