arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1912.07045cs.AI
收藏

代理人应该如何实践?

How Should an Agent Practice?

University of Michigan(密歇根大学)

Janarthanan Rajendran, Richard Lewis, Vivek Veeriah, Honglak Lee, Satinder Singh

摘要:

我们提出了一种方法,用于学习内在奖励函数,以驱动学习的代理在实践期间,外在任务奖励不可用。在实践过程中,环境可能与外部奖励的培训和评估不同。我们把这种练习和客观评价的交替阶段称为练习比赛,类比于人类在体育和游戏中常见的技能获得机制。经纪人必须有效地利用练习环境中的时间段,以便在比赛中提高表现。在所提出的方法中,通过元梯度方法来学习内在的实践奖励,该方法调整实践奖励参数以减少从匹配计算的外在匹配奖励损失。我们说明了一个简单的网格世界上的方法,并评估它在两个游戏中的实践环境不同的比赛:乒乓球与实践对墙没有对手,和吃豆人与实践中的迷宫没有鬼。结果表明,除了比赛期间的学习,在实践中学习的收益只在比赛。

英文摘要:

We present a method for learning intrinsic reward functions to drive the learning of an agent during periods of practice in which extrinsic task rewards are not available. During practice, the environment may differ from the one available for training and evaluation with extrinsic rewards. We refer to this setup of alternating periods of practice and objective evaluation as practice-match, drawing an analogy to regimes of skill acquisition common for humans in sports and games. The agent must effectively use periods in the practice environment so that performance improves during matches. In the proposed method the intrinsic practice reward is learned through a meta-gradient approach that adapts the practice reward parameters to reduce the extrinsic match reward loss computed from matches. We illustrate the method on a simple grid world, and evaluate it in two games in which the practice environment differs from match: Pong with practice against a wall without an opponent, and PacMan with practice in a maze without ghosts. The results show gains from learning in practice in addition to match periods over learning in matches only.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑