什么是好的决策扩散规划器？
What Makes a Good Diffusion Planner for Decision Making?
AI总结：
通过在离线强化学习下训练评估超6000个扩散模型，系统研究了扩散规划的关键组件，发现反常规设计能提升性能，并据此提出了在标准基准上达到最先进水平的基线。
摘要：
扩散模型最近在解决决策问题中展现了巨大潜力，特别是在生成行为规划（也称为扩散规划）方面。尽管许多研究证明了扩散规划的出色性能，但一个好的扩散规划器关键组件背后的机制仍不清楚，且现有研究中的设计选择高度不一致。在本研究中，我们通过在离线强化学习（RL）设置下对扩散规划进行系统的实证实验来解决这一问题，为扩散规划的核心组件提供实用见解。我们训练并评估了超过6,000个扩散模型，识别出诸如引导采样、网络架构、动作生成和规划策略等关键组件。我们发现，与以往扩散规划工作中的常见做法相反的一些设计选择实际上能带来更好的性能，例如，带选择的无条件采样可能优于引导采样，且Transformer作为去噪网络的表现优于U-Net。基于这些见解，我们提出了一种简单而强大的扩散规划基线，在标准离线RL基准上取得了最先进的结果。
英文摘要：
Diffusion models have recently shown significant potential in solving decision-making problems, particularly in generating behavior plans -- also known as diffusion planning. While numerous studies have demonstrated the impressive performance of diffusion planning, the mechanisms behind the key components of a good diffusion planner remain unclear and the design choices are highly inconsistent in existing studies. In this work, we address this issue through systematic empirical experiments on diffusion planning in an offline reinforcement learning (RL) setting, providing practical insights into the essential components of diffusion planning. We trained and evaluated over 6,000 diffusion models, identifying the critical components such as guided sampling, network architecture, action generation and planning strategy. We revealed that some design choices opposite to the common practice in previous work in diffusion planning actually lead to better performance, e.g., unconditional sampling with selection can be better than guided sampling and Transformer outperforms U-Net as denoising network. Based on these insights, we suggest a simple yet strong diffusion planning baseline that achieves state-of-the-art results on standard offline RL benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
