利用扩散模型恢复含噪演示以进行模仿学习
Restoring Noisy Demonstration for Imitation Learning With Diffusion Models
AI总结：
本文提出一种过滤-恢复框架，利用条件扩散模型恢复含噪专家演示，在机器人操作和移动任务中优于现有方法。
摘要：
模仿学习（IL）旨在从专家演示中学习策略，并已被应用于各种场景。通过向专家策略学习，模仿学习方法无需环境交互或奖励信号。然而，大多数现有的模仿学习算法假设专家演示是完美的，但专家演示常常包含由人类专家错误或传感器/控制系统不精确导致的缺陷。为解决上述问题，本工作提出了一种过滤-恢复框架，以最佳方式利用带有固有噪声的专家演示。我们提出的方法首先从演示中过滤出干净样本，然后学习条件扩散模型以恢复含噪样本。我们在多个领域评估了我们提出的框架和现有方法，包括机器人臂操作、灵巧操作和移动。实验结果表明，我们提出的框架在所有任务中均持续优于现有方法。消融研究进一步验证了每个组件的有效性，并展示了该框架对不同噪声类型和水平的鲁棒性。这些结果证实了我们的框架对含噪离线演示数据的实际适用性。
英文摘要：
Imitation learning (IL) aims to learn a policy from expert demonstrations and has been applied to various applications. By learning from the expert policy, IL methods do not require environmental interactions or reward signals. However, most existing imitation learning algorithms assume perfect expert demonstrations, but expert demonstrations often contain imperfections caused by errors from human experts or sensor/control system inaccuracies. To address the above problems, this work proposes a filter-and-restore framework to best leverage expert demonstrations with inherent noise. Our proposed method first filters clean samples from the demonstrations and then learns conditional diffusion models to recover the noisy ones. We evaluate our proposed framework and existing methods in various domains, including robot arm manipulation, dexterous manipulation, and locomotion. The experiment results show that our proposed framework consistently outperforms existing methods across all the tasks. Ablation studies further validate the effectiveness of each component and demonstrate the framework's robustness to different noise types and levels. These results confirm the practical applicability of our framework to noisy offline demonstration data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
