SD2AIL: 通过扩散模型从合成演示中进行对抗模仿学习
SD2AIL: Adversarial Imitation Learning from Synthetic Demonstrations via Diffusion Models
AI总结：
本文提出SD2AIL，利用扩散模型生成合成演示作为伪专家数据，结合优先级专家演示回放策略提升性能，实验显示在Hopper任务中优于现有方法。
摘要：
对抗模仿学习（AIL）是模仿学习中的主导框架，通过专家演示推断奖励以指导策略优化。尽管提供更多专家演示通常能提升性能和稳定性，但在某些场景下收集此类演示可能具有挑战性。受扩散模型在数据生成中的成功启发，我们提出SD2AIL，利用扩散模型生成合成演示。我们首先使用扩散模型在判别器中生成合成演示作为伪专家数据以增强专家演示。为进一步选择性地回放大量（伪）专家演示中最有价值的演示，我们进一步引入优先级专家演示回放策略（PEDR）。在模拟任务上的实验结果证明了我们方法的有效性和鲁棒性。特别是，在Hopper任务中，我们的方法实现了平均回报3441，超过现有最先进方法89。我们的代码将在https://github.com/positron-lpc/SD2AIL上提供。
英文摘要：
Adversarial Imitation Learning (AIL) is a dominant framework in imitation learning that infers rewards from expert demonstrations to guide policy optimization. Although providing more expert demonstrations typically leads to improved performance and greater stability, collecting such demonstrations can be challenging in certain scenarios. Inspired by the success of diffusion models in data generation, we propose SD2AIL, which utilizes synthetic demonstrations via diffusion models. We first employ a diffusion model in the discriminator to generate synthetic demonstrations as pseudo-expert data that augment the expert demonstrations. To selectively replay the most valuable demonstrations from the large pool of (pseudo-) expert demonstrations, we further introduce a prioritized expert demonstration replay strategy (PEDR). The experimental results on simulation tasks demonstrate the effectiveness and robustness of our method. In particular, in the Hopper task, our method achieves an average return of 3441, surpassing the state-of-the-art method by 89. Our code will be available at https://github.com/positron-lpc/SD2AIL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
