相似地看，相似地做：异质可观察的模仿学习
Seeing Differently, Acting Similarly: Heterogeneously Observable Imitation Learning
摘要：
在许多真实世界的模仿学习任务中，演示者和学习者必须在不同的观察空间下行动。这种情况给现有的模仿学习方法带来了很大的障碍，因为它们大多是在同质观测空间下学习策略的。另一方面，以前的研究在不同的观察空间有很强的假设，这两个观察空间共存的整个学习过程中。然而，在现实中，由于获取专家观测的高成本，观测共存将受到限制。在这项工作中，我们研究了这个具有挑战性的问题，有限的观察共存下异质观测：异质可观察模仿学习（HOIL）。我们确定了HOIL中的两个潜在问题：动力学失配和支持失配，并进一步提出了基于重要性加权和拒绝学习的重要性加权拒绝（IWRE）算法来解决HOIL问题。实验结果表明，IWRE可以解决各种HOIL任务，包括具有挑战性的任务，转换基于视觉的演示，以随机存取存储器（RAM）为基础的政策在雅达利域，即使有限的视觉观察。
英文摘要：
In many real-world imitation learning tasks, the demonstrator and the learner have to act under different observation spaces. This situation brings significant obstacles to existing imitation learning approaches, since most of them learn policies under homogeneous observation spaces. On the other hand, previous studies under different observation spaces have strong assumptions that these two observation spaces coexist during the entire learning process. However, in reality, the observation coexistence will be limited due to the high cost of acquiring expert observations. In this work, we study this challenging problem with limited observation coexistence under heterogeneous observations: Heterogeneously Observable Imitation Learning (HOIL). We identify two underlying issues in HOIL: the dynamics mismatch and the support mismatch, and further propose the Importance Weighting with REjection (IWRE) algorithm based on importance weighting and learning with rejection to solve HOIL problems. Experimental results show that IWRE can solve various HOIL tasks, including the challenging tasks of transforming the vision-based demonstrations to random access memory (RAM)-based policies in the Atari domain, even with limited visual observations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
