探索性模仿学习:面向连续环境的路径签名方法
Explorative Imitation Learning: A Path Signature Approach for Continuous Environments
AI总结:
提出CILO方法,结合探索与路径签名,减少专家轨迹需求并自动编码约束,在五个环境中取得最优性能,其中两个环境超越专家。
摘要:
一些模仿学习方法将行为克隆与自监督相结合,以从状态对中推断动作。然而,大多数方法依赖大量专家轨迹来提升泛化能力,并需要人工干预以捕捉问题的关键方面,如领域约束。在本文中,我们提出了连续环境观察模仿学习(CILO),这是一种新的模仿学习方法,增强了两个重要特性:(i)探索,允许更多样化的状态转换,减少对专家轨迹的需求,并导致更少的训练迭代;(ii)路径签名,通过创建智能体和专家轨迹的非参数表示,实现约束的自动编码。我们将CILO与一个基线方法和两种领先的模仿学习方法在五个环境中进行了比较。它在所有环境中的整体性能均优于所有方法,并在其中两个环境中超越了专家表现。
英文摘要:
Some imitation learning methods combine behavioural cloning with self-supervision to infer actions from state pairs. However, most rely on a large number of expert trajectories to increase generalisation and human intervention to capture key aspects of the problem, such as domain constraints. In this paper, we propose Continuous Imitation Learning from Observation (CILO), a new method augmenting imitation learning with two important features: (i) exploration, allowing for more diverse state transitions, requiring less expert trajectories and resulting in fewer training iterations; and (ii) path signatures, allowing for automatic encoding of constraints, through the creation of non-parametric representations of agents and expert trajectories. We compared CILO with a baseline and two leading imitation learning methods in five environments. It had the best overall performance of all methods in all environments, outperforming the expert in two of them.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。