通过变分自适应加权的快速稳定扩散规划
Fast and Stable Diffusion Planning through Variational Adaptive Weighting
AI总结：
本文提出一种变分最优的不确定性感知加权函数及其闭式多项式逼近方法，用于扩散规划中的自适应损失加权，在Maze2D和Kitchen基准上以少10倍的训练步骤达到竞争性能。
摘要：
扩散模型最近在离线强化学习中显示出潜力。然而，这些方法常常遭受高训练成本和缓慢收敛的困扰，尤其是在使用基于Transformer的去噪骨干网络时。尽管已经提出了几种优化策略——例如修改噪声调度、辅助预测目标和自适应损失加权——但在实现稳定高效的训练方面仍存在挑战。特别是，现有的损失加权函数通常依赖神经网络逼近器，这在早期训练阶段可能无效，因为MLP在早期训练阶段面对稀疏反馈时泛化能力有限。在这项工作中，我们推导出一个变分最优的不确定性感知加权函数，并引入一种闭式多项式逼近方法，用于在基于流的生成建模框架下进行其在线估计。我们将我们的方法集成到扩散规划流程中，并在标准离线强化学习基准上进行评估。在Maze2D和Kitchen任务上的实验结果表明，我们的方法在训练步骤减少多达10倍的情况下实现了有竞争力的性能，突显了其实际有效性。
英文摘要：
Diffusion models have recently shown promise in offline RL. However, these methods often suffer from high training costs and slow convergence, particularly when using transformer-based denoising backbones. While several optimization strategies have been proposed -- such as modified noise schedules, auxiliary prediction targets, and adaptive loss weighting -- challenges remain in achieving stable and efficient training. In particular, existing loss weighting functions typically rely on neural network approximators, which can be ineffective in early training phases due to limited generalization capacity of MLPs when exposed to sparse feedback in the early training stages. In this work, we derive a variationally optimal uncertainty-aware weighting function and introduce a closed-form polynomial approximation method for its online estimation under the flow-based generative modeling framework. We integrate our method into a diffusion planning pipeline and evaluate it on standard offline RL benchmarks. Experimental results on Maze2D and Kitchen tasks show that our method achieves competitive performance with up to 10 times fewer training steps, highlighting its practical effectiveness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
