使用AlphaZero-likeRanked奖励强化学习解决Morpion Solitaire问题
Tackling Morpion Solitaire with AlphaZero-likeRanked Reward Reinforcement Learning
摘要：
Morpion Solitaire是一款流行的单人游戏，用纸和铅笔进行。由于其大的状态空间（在围棋游戏的顺序），传统的搜索算法，如MCTS，一直无法找到好的解决方案。后来的算法，嵌套推出政策适应，能够找到一个新的记录82个步骤，虽然有大量的计算资源。据我们所知，在取得这一记录之后，大约十年来没有进一步进展的报告。在本文中，我们将AlphaGo/AlphaZero的深度自学习强化学习方法最近令人印象深刻的表现作为设计Morpion Solitaire的启发。Morpion Solitaire的一个挑战是状态空间是稀疏的，几乎没有赢/输信号。相反，我们使用一种称为排名奖励的方法来为Morpion Solitaire创建一个强化学习自我游戏框架。这使我们能够找到中等质量的解决方案与合理的计算工作。我们的记录是一个67步的解决方案，这非常接近人类最好的解决方案（68），除了使用排名奖励之外，没有任何其他适应问题的方法。我们列出了许多进一步的潜在改进途径。
英文摘要：
Morpion Solitaire is a popular single player game, performed with paper and pencil. Due to its large state space (on the order of the game of Go) traditional search algorithms, such as MCTS, have not been able to find good solutions. A later algorithm, Nested Rollout Policy Adaptation, was able to find a new record of 82 steps, albeit with large computational resources. After achieving this record, to the best of our knowledge, there has been no further progress reported, for about a decade. In this paper we take the recent impressive performance of deep self-learning reinforcement learning approaches from AlphaGo/AlphaZero as inspiration to design a searcher for Morpion Solitaire. A challenge of Morpion Solitaire is that the state space is sparse, there are few win/loss signals. Instead, we use an approach known as ranked reward to create a reinforcement learning self-play framework for Morpion Solitaire. This enables us to find medium-quality solutions with reasonable computational effort. Our record is a 67 steps solution, which is very close to the human best (68) without any other adaptation to the problem than using ranked reward. We list many further avenues for potential improvement.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
