arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1202.1558cs.LG
收藏

最大似然逆强化学习的性能研究

On the Performance of Maximum Likelihood Inverse Reinforcement Learning

Universidad de Zaragoza(萨拉戈萨大学);Centro Universitario de la Defensa, Zaragoza(萨拉戈萨国防大学中心)

Héctor Ratia, Luis Montesano, Ruben Martinez-Cantin

摘要:

逆强化学习(IRL)解决了恢复任务描述的问题,给出了用于解决此类任务的最佳策略的演示。最优策略通常由专家或教师提供,使得IRL特别适合于学徒学习问题。任务描述以马尔可夫决策过程(MDP)的奖励函数的形式编码。已经提出了几种算法来找到对应于一组演示的奖励函数。在不同应用中提供最佳结果的算法之一是优化策略平方误差准则的梯度方法。在平行线的研究,其他作者最近提出了一个梯度近似的最大似然估计的奖励信号。一般来说,这两种方法在不同阶段近似梯度估计和准则,使算法易于处理和有效。在这项工作中,我们提供了一个详细的描述不同的方法,以突出奖励估计,政策相似性和计算成本方面的差异。我们还提供了实验结果,以评估性能的差异的方法。

英文摘要:

Inverse reinforcement learning (IRL) addresses the problem of recovering a task description given a demonstration of the optimal policy used to solve such a task. The optimal policy is usually provided by an expert or teacher, making IRL specially suitable for the problem of apprenticeship learning. The task description is encoded in the form of a reward function of a Markov decision process (MDP). Several algorithms have been proposed to find the reward function corresponding to a set of demonstrations. One of the algorithms that has provided best results in different applications is a gradient method to optimize a policy squared error criterion. On a parallel line of research, other authors have presented recently a gradient approximation of the maximum likelihood estimate of the reward signal. In general, both approaches approximate the gradient estimate and the criteria at different stages to make the algorithm tractable and efficient. In this work, we provide a detailed description of the different methods to highlight differences in terms of reward estimation, policy similarity and computational costs. We also provide experimental results to evaluate the differences in performance of the methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑