arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.06956cs.LGcs.AI
收藏

去相关双Q学习

Decorrelated Double Q-learning

SUNY at Buffalo(纽约州立大学布法罗分校)

Gang Chen

摘要:

基于价值函数逼近的Q学习算法由于存在高估偏差和不精确估计而导致学习效果不佳。具体而言,高估偏差来自噪声估计上的最大算子,其使用后续状态的估计被夸大。受深度强化学习和双Q学习的最新进展的启发,我们引入了去相关双Q学习(D2Q)。具体来说,我们引入去相关正则化项来减少价值函数逼近器之间的相关性,这可以导致更少的有偏估计和更低的方差。一组MuJoCo连续控制任务的实验结果表明,我们的去相关双Q学习可以有效地提高性能。

英文摘要:

Q-learning with value function approximation may have the poor performance because of overestimation bias and imprecise estimate. Specifically, overestimation bias is from the maximum operator over noise estimate, which is exaggerated using the estimate of a subsequent state. Inspired by the recent advance of deep reinforcement learning and Double Q-learning, we introduce the decorrelated double Q-learning (D2Q). Specifically, we introduce the decorrelated regularization item to reduce the correlation between value function approximators, which can lead to less biased estimation and low variance. The experimental results on a suite of MuJoCo continuous control tasks demonstrate that our decorrelated double Q-learning can effectively improve the performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑