arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2010.08353cs.ROcs.LG
收藏

观察模仿学习与演示学习的保证几乎等价性

On the Guaranteed Almost Equivalence between Imitation Learning from Observation and Demonstration

The University of Sydney(悉尼大学);Beihang University(北京航空航天大学);The Chinese University of Hong Kong(香港中文大学);Tencent Robotics X(腾讯机器人X实验室)

Zhihao Cheng, Liu Liu, Aishan Liu, Hao Sun, Meng Fang, Dacheng Tao

摘要:

从观察的模仿学习(LfO)比从示范的模仿学习(LfD)更可取,因为在从专家数据重构专家策略时不需要专家动作。然而,以往的研究表明,LfO的性能是劣于LfD的巨大差距,这使得它具有挑战性,在实践中使用LfO。通过对比,本文证明了LfO几乎等价于LfD在确定性的机器人环境中,更一般地,即使在机器人环境中有界随机性。在确定性的机器人环境中,从控制理论的角度来看,我们表明,LfO和LfD之间的逆动力学不一致接近零,这意味着LfO几乎相当于LfD。为了进一步放松确定性约束,更好地适应实际环境,我们考虑了机器人环境中的有界随机性,并证明了LfD和LfO的优化目标在更广义的设置中几乎保持相同。多个机器人任务进行了大量的实验,以经验证明,LfO实现可比的性能LfD。实际上,现实中最常见的机器人系统是具有有界随机性的机器人环境(即,本文所考虑的环境)。因此,我们的研究结果大大扩展了LfO的潜力,并表明我们可以安全地应用LfO,而不会牺牲LfD在实践中的性能。

英文摘要:

Imitation learning from observation (LfO) is more preferable than imitation learning from demonstration (LfD) due to the nonnecessity of expert actions when reconstructing the expert policy from the expert data. However, previous studies imply that the performance of LfO is inferior to LfD by a tremendous gap, which makes it challenging to employ LfO in practice. By contrast, this paper proves that LfO is almost equivalent to LfD in the deterministic robot environment, and more generally even in the robot environment with bounded randomness. In the deterministic robot environment, from the perspective of the control theory, we show that the inverse dynamics disagreement between LfO and LfD approaches zero, meaning that LfO is almost equivalent to LfD. To further relax the deterministic constraint and better adapt to the practical environment, we consider bounded randomness in the robot environment and prove that the optimizing targets for both LfD and LfO remain almost same in the more generalized setting. Extensive experiments for multiple robot tasks are conducted to empirically demonstrate that LfO achieves comparable performance to LfD. In fact, most common robot systems in reality are the robot environment with bounded randomness (i.e., the environment this paper considered). Hence, our findings greatly extend the potential of LfO and suggest that we can safely apply LfO without sacrificing the performance compared to LfD in practice.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑