arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1210.4876cs.LGstat.ML
收藏

基于I.I.D.化归的主动模仿学习主动学习

Active Imitation Learning via Reduction to I.I.D. Active Learning

Oregon State University(俄勒冈州立大学)

Kshitij Judah, Alan Fern, Thomas G. Dietterich

摘要:

在标准的被动模仿学习中,目标是通过被动地观察目标策略的完整执行轨迹来学习目标策略。不幸的是,生成这样的轨迹可能需要大量的专家努力,并且在某些情况下是不切实际的。在本文中,我们认为主动模仿学习的目标是减少这种努力,通过询问专家所需的行动在个别国家,这是根据过去的查询和学习者的互动与环境模拟器的答案选择。我们介绍了一种新的方法,基于减少主动模仿学习i.i.d.。主动学习,可以利用i.i.d.的进步。设置.我们的第一个贡献,是分析非静态和静态策略的减少,表明主动模仿学习的标签复杂性(查询数量)可以大大低于被动学习。我们的第二个贡献,是引入一个实用的算法的启发减少,这是非常有效的,在四个测试域相比,一些替代品。

英文摘要:

In standard passive imitation learning, the goal is to learn a target policy by passively observing full execution trajectories of it. Unfortunately, generating such trajectories can require substantial expert effort and be impractical in some cases. In this paper, we consider active imitation learning with the goal of reducing this effort by querying the expert about the desired action at individual states, which are selected based on answers to past queries and the learner's interactions with an environment simulator. We introduce a new approach based on reducing active imitation learning to i.i.d. active learning, which can leverage progress in the i.i.d. setting. Our first contribution, is to analyze reductions for both non-stationary and stationary policies, showing that the label complexity (number of queries) of active imitation learning can be substantially less than passive learning. Our second contribution, is to introduce a practical algorithm inspired by the reductions, which is shown to be highly effective in four test domains compared to a number of alternatives.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑