arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1206.5264cs.LGstat.ML
收藏

使用逆强化学习和梯度方法的学徒学习

Apprenticeship Learning using Inverse Reinforcement Learning and Gradient Methods

Budapest University of Technology and Economics(布达佩斯技术与经济大学);University of Alberta(阿尔伯塔大学);Computer and Automation Research Institute of the Hungarian Academy of Sciences(匈牙利科学院计算机与自动化研究所)

Gergely Neu, Csaba Szepesvari

摘要:

在本文中,我们提出了一种新的梯度算法来学习一个政策,从专家的观察行为假设专家的行为最优相对于一些未知的奖励函数的马尔可夫决策问题。该算法的目标是找到一个奖励函数,使得得到的最优策略与专家的观察行为相匹配。主要的困难是,从参数到政策的映射是不光滑和高度冗余的。利用次微分解决了第一个困难,而通过计算自然梯度克服了第二个困难。我们在两个人工域中测试了所提出的方法,发现它比以前的一些方法更可靠和有效。

英文摘要:

In this paper we propose a novel gradient algorithm to learn a policy from an expert's observed behavior assuming that the expert behaves optimally with respect to some unknown reward function of a Markovian Decision Problem. The algorithm's aim is to find a reward function such that the resulting optimal policy matches well the expert's observed behavior. The main difficulty is that the mapping from the parameters to policies is both nonsmooth and highly redundant. Resorting to subdifferentials solves the first difficulty, while the second one is over- come by computing natural gradients. We tested the proposed method in two artificial domains and found it to be more reliable and efficient than some previous methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑