arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1802.08842cs.NE
收藏

回到基础:基准规范演化策略玩雅达利

Back to Basics: Benchmarking Canonical Evolution Strategies for Playing Atari

Patryk Chrabaszcz, Ilya Loshchilov, Frank Hutter

摘要:

演化策略(ES)最近被证明是强化学习(RL)算法在一组具有挑战性的深度RL问题上的可行替代方案,包括Atari游戏和MuJoCo人形运动基准。虽然在这项工作中的ES算法属于专业类的自然演化策略(类似于近似梯度RL算法,如REINFORCE),我们证明,即使是一个非常基本的规范ES算法可以实现相同的,甚至更好的性能。一个基本的ES算法的成功表明,国家的最先进的,可以进一步通过整合在过去几十年中在ES领域取得的许多进展。我们还定性地证明了ES算法具有与传统RL算法非常不同的性能特征:在某些游戏中,它们学会利用环境并表现得更好,而在其他游戏中,它们可能会陷入次优局部极小值。因此,将它们的优势与传统RL算法的优势相结合,可能会导致最新技术的新进展。

英文摘要:

Evolution Strategies (ES) have recently been demonstrated to be a viable alternative to reinforcement learning (RL) algorithms on a set of challenging deep RL problems, including Atari games and MuJoCo humanoid locomotion benchmarks. While the ES algorithms in that work belonged to the specialized class of natural evolution strategies (which resemble approximate gradient RL algorithms, such as REINFORCE), we demonstrate that even a very basic canonical ES algorithm can achieve the same or even better performance. This success of a basic ES algorithm suggests that the state-of-the-art can be advanced further by integrating the many advances made in the field of ES in the last decades. We also demonstrate qualitatively that ES algorithms have very different performance characteristics than traditional RL algorithms: on some games, they learn to exploit the environment and perform much better while on others they can get stuck in suboptimal local minima. Combining their strengths with those of traditional RL algorithms is therefore likely to lead to new advances in the state of the art.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑