arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.04678cs.LGstat.ML
收藏

原始沃瑟斯坦模仿学习

Primal Wasserstein Imitation Learning

Google Research, Brain Team(谷歌研究院大脑团队);Univ. de Lille(里尔大学);CNRS(法国国家科学研究中心);Inria(法国国家信息与自动化研究所);UMR 9189 CRIStAL

Robert Dadashi, Léonard Hussenot, Matthieu Geist, Olivier Pietquin

摘要:

模仿学习(IL)方法试图将智能体的行为与专家的行为相匹配。在目前的工作中,我们提出了一种新的IL方法的基础上,一个概念上简单的算法:原始Wasserstein模仿学习(PWIL),它的关系到原始形式的Wasserstein距离之间的专家和代理状态动作分布。我们提出了一个离线导出的奖励函数,而不是最近的对抗IL算法,通过与环境的交互来学习奖励函数,并且几乎不需要微调。我们表明,我们可以恢复专家的行为在各种连续的控制任务的MuJoCo域在一个样本有效的方式在代理交互和专家与环境的交互。最后,我们表明,代理人的行为,我们训练匹配的行为与专家的Wasserstein距离,而不是常用的代理性能。

英文摘要:

Imitation Learning (IL) methods seek to match the behavior of an agent with that of an expert. In the present work, we propose a new IL method based on a conceptually simple algorithm: Primal Wasserstein Imitation Learning (PWIL), which ties to the primal form of the Wasserstein distance between the expert and the agent state-action distributions. We present a reward function which is derived offline, as opposed to recent adversarial IL algorithms that learn a reward function through interactions with the environment, and which requires little fine-tuning. We show that we can recover expert behavior on a variety of continuous control tasks of the MuJoCo domain in a sample efficient manner in terms of agent interactions and of expert interactions with the environment. Finally, we show that the behavior of the agent we train matches the behavior of the expert with the Wasserstein distance, rather than the commonly used proxy of performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑