arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1905.02841cs.LGmath.OCstat.ML
收藏

Q-learning的加速目标更新

Accelerated Target Updates for Q-learning

Bowen Weng, Huaqing Xiong, Wei Zhang

摘要:

研究了Q-学习算法中的加速问题。我们提出了一个加速目标更新方案,结合历史迭代的Q函数。这个想法在概念上受到最优化理论中基于动量的加速方法的启发。所提出的加速算法收敛的条件下建立。这些算法使用强化学习中常用的测试问题进行验证,包括FrozenLake网格世界游戏,Deepmind Control Suite中的两个离散时间LQR问题和Atari 2600游戏。仿真结果表明,与普通Q-学习算法相比,本文提出的加速算法可以提高收敛性能。

英文摘要:

This paper studies accelerations in Q-learning algorithms. We propose an accelerated target update scheme by incorporating the historical iterates of Q functions. The idea is conceptually inspired by the momentum-based accelerated methods in the optimization theory. Conditions under which the proposed accelerated algorithms converge are established. The algorithms are validated using commonly adopted testing problems in reinforcement learning, including the FrozenLake grid world game, two discrete-time LQR problems from the Deepmind Control Suite, and the Atari 2600 games. Simulation results show that the proposed accelerated algorithms can improve the convergence performance compared with the vanilla Q-learning algorithm.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑