DIAR：基于扩散模型引导并带有自适应重估的隐式 Q 学习
DIAR: Diffusion-model-guided Implicit Q-learning with Adaptive Revaluation
AI总结：
本文提出 DIAR 框架，通过扩散模型学习序列分布并引入自适应重估机制动态调整决策长度，有效解决了离线强化学习中的分布外样本和长时序问题，在稀疏奖励环境中表现优异。
摘要：
我们提出了一种新颖的离线强化学习方法，引入了基于扩散模型引导并带有自适应重估的隐式 Q 学习（DIAR）框架。我们解决了离线强化学习中的两个关键挑战：分布外样本和长时序问题。我们利用扩散模型学习状态-动作序列分布，并结合价值函数以实现更平衡和自适应的决策。DIAR 引入了自适应重估机制，通过比较当前和未来的状态值来动态调整决策长度，从而实现灵活的长期决策。此外，我们将 Q 网络学习与扩散模型引导的价值函数相结合，解决了 Q 值高估问题。扩散模型生成多样的潜在轨迹，增强了策略的鲁棒性和泛化能力。正如在 Maze2D、AntMaze 和 Kitchen 等任务中所展示的，DIAR 在长时序、稀疏奖励环境中始终优于最先进的算法。
英文摘要：
We propose a novel offline reinforcement learning (offline RL) approach, introducing the Diffusion-model-guided Implicit Q-learning with Adaptive Revaluation (DIAR) framework. We address two key challenges in offline RL: out-of-distribution samples and long-horizon problems. We leverage diffusion models to learn state-action sequence distributions and incorporate value functions for more balanced and adaptive decision-making. DIAR introduces an Adaptive Revaluation mechanism that dynamically adjusts decision lengths by comparing current and future state values, enabling flexible long-term decision-making. Furthermore, we address Q-value overestimation by combining Q-network learning with a value function guided by a diffusion model. The diffusion model generates diverse latent trajectories, enhancing policy robustness and generalization. As demonstrated in tasks like Maze2D, AntMaze, and Kitchen, DIAR consistently outperforms state-of-the-art algorithms in long-horizon, sparse-reward environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
