arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2110.15331cs.LGcs.AI
收藏

Wasserstein距离最大化内在控制

Wasserstein Distance Maximizing Intrinsic Control

The University of Texas at Austin(德克萨斯大学奥斯汀分校);DeepMind

Ishan Durugkar, Steven Hansen, Stephen Spencer, Volodymyr Mnih

摘要:

本文讨论的问题,学习技能的条件政策,有意义的行为在没有奖励信号。基于互信息的目标已经在学习技能方面显示出一些成功,这些技能在这种设置中达到不同的状态集合。这些目标包括KL-发散项,其通过访问不同的状态来最大化,即使这些状态在MDP中相距不远。本文提出了一种方法,奖励智能体学习技能,最大限度地提高他们的状态访问从开始状态的技能的Wasserstein距离。它表明,这样的目标导致的政策,覆盖更多的距离在MDP比多样性为基础的目标,并验证了各种雅达利环境的结果。

英文摘要:

This paper deals with the problem of learning a skill-conditioned policy that acts meaningfully in the absence of a reward signal. Mutual information based objectives have shown some success in learning skills that reach a diverse set of states in this setting. These objectives include a KL-divergence term, which is maximized by visiting distinct states even if those states are not far apart in the MDP. This paper presents an approach that rewards the agent for learning skills that maximize the Wasserstein distance of their state visitation from the start state of the skill. It shows that such an objective leads to a policy that covers more distance in the MDP than diversity based objectives, and validates the results on a variety of Atari environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑