连续均值零分歧正则化模仿学习(CMZ-DRIL)
Continuous Mean-Zero Disagreement-Regularized Imitation Learning (CMZ-DRIL)
AI总结:
针对少量专家示范且无环境奖励的模仿学习问题,提出CMZ-DRIL,利用智能体集成动作分歧构造连续均值零奖励并强化优化,在导航和MuJoCo任务中优于既有方法。
摘要:
模仿学习和强化学习等机器学习范式能够在多种复杂环境中生成高性能智能体。然而,常用方法需要大量数据和/或已知奖励函数。本文提出一种名为连续均值零分歧正则化模仿学习(CMZ-DRIL)的方法,该方法采用一种新颖的奖励结构,以提升仅能获取少量专家示范的模仿学习智能体的性能。CMZ-DRIL使用强化学习来最小化一组经过训练以建模专家示范的智能体集成之间的不确定性。该方法不使用任何特定于环境的奖励,而是根据智能体集成的动作分歧创建一个连续且均值为零的奖励函数。正如在一个航点导航环境和两个MuJoCo环境中所证明的那样,CMZ-DRIL能够生成高性能智能体,在若干关键指标上,这些智能体的行为比此前主要方法更接近专家。
英文摘要:
Machine-learning paradigms such as imitation learning and reinforcement learning can generate highly performant agents in a variety of complex environments. However, commonly used methods require large quantities of data and/or a known reward function. This paper presents a method called Continuous Mean-Zero Disagreement-Regularized Imitation Learning (CMZ-DRIL) that employs a novel reward structure to improve the performance of imitation-learning agents that have access to only a handful of expert demonstrations. CMZ-DRIL uses reinforcement learning to minimize uncertainty among an ensemble of agents trained to model the expert demonstrations. This method does not use any environment-specific rewards, but creates a continuous and mean-zero reward function from the action disagreement of the agent ensemble. As demonstrated in a waypoint-navigation environment and in two MuJoCo environments, CMZ-DRIL can generate performant agents that behave more similarly to the expert than primary previous approaches in several key metrics.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。