从无标签和含噪演示中进行程序化模仿学习
Programmatic Imitation Learning from Unlabeled and Noisy Demonstrations
AI总结：
本文提出 PLUNDER 算法，通过将概率程序合成器集成到迭代期望最大化框架中，从无标签且含噪的演示中学习可解释、可修复的程序化策略，并在多个任务上显著优于现有模仿学习方法。
摘要：
模仿学习（IL）是一种通过演示来教机器人执行新任务的有前景的范式。现有的大多数 IL 方法使用神经网络（NN），但这些方法存在几个众所周知的局限：1）需要大量训练数据，2）难以解释，3）难以修复和适应。程序化模仿学习（PIL）正受到越来越多的关注，它在解决上述局限方面具有显著潜力。在 PIL 中，学习到的策略用编程语言表示，使其易于解释和修复。然而，最先进的 PIL 算法假设能够获得动作标签，并且难以从含噪的真实世界演示中学习。在本文中，我们提出 PLUNDER，一种新颖的 PIL 算法，它将概率程序合成器集成到迭代期望最大化（EM）框架中，以解决这些不足。与现有的 PIL 方法不同，PLUNDER 合成的概率程序化策略特别适合对真实世界演示中固有的不确定性进行建模。我们的方法利用 EM 循环同时推断缺失的动作标签和最可能的概率策略。我们将 PLUNDER 与几种已有的 IL 技术进行了基准比较，并在五个具有挑战性的含噪模仿学习任务上证明了其优越性。PLUNDER 策略在匹配给定演示方面达到 95% 的准确率，比次优基线高出 19%。此外，PLUNDER 生成的策略成功完成任务的频率比最接近的基线高出 17%。
英文摘要：
Imitation Learning (IL) is a promising paradigm for teaching robots to perform novel tasks using demonstrations. Most existing approaches for IL utilize neural networks (NN), however, these methods suffer from several well-known limitations: they 1) require large amounts of training data, 2) are hard to interpret, and 3) are hard to repair and adapt. There is an emerging interest in programmatic imitation learning (PIL), which offers significant promise in addressing the above limitations. In PIL, the learned policy is represented in a programming language, making it amenable to interpretation and repair. However, state-of-the-art PIL algorithms assume access to action labels and struggle to learn from noisy real-world demonstrations. In this paper, we propose PLUNDER, a novel PIL algorithm that integrates a probabilistic program synthesizer in an iterative Expectation-Maximization (EM) framework to address these shortcomings. Unlike existing PIL approaches, PLUNDER synthesizes probabilistic programmatic policies that are particularly well-suited for modeling the uncertainties inherent in real-world demonstrations. Our approach leverages an EM loop to simultaneously infer the missing action labels and the most likely probabilistic policy. We benchmark PLUNDER against several established IL techniques, and demonstrate its superiority across five challenging imitation learning tasks under noise. PLUNDER policies achieve 95% accuracy in matching the given demonstrations, outperforming the next best baseline by 19%. Additionally, policies generated by PLUNDER successfully complete the tasks 17% more frequently than the nearest baseline.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
