学习停止：运动仿真蒙特卡罗树搜索
Learning to Stop: Dynamic Simulation Monte-Carlo Tree Search
摘要：
蒙特卡罗树搜索（MCTS）在与深度神经网络（DNN）结合时，在许多领域（如围棋和雅达利游戏）中取得了最先进的结果。当执行更多模拟时，MCTS可以实现更高的性能，但也需要大量的CPU和GPU资源。然而，并不是所有的状态都需要很长的搜索时间来确定代理可以找到的最佳操作。例如，在19 x19 Go和NoGo中，我们发现对于超过一半的状态，DNN预测的最佳动作即使在搜索2分钟后也保持不变。这意味着，如果我们能够在对当前搜索结果有信心时更早地停止搜索，则可以节省大量的资源。在本文中，我们提出通过预测当前搜索状态的不确定性来实现这一目标，并使用结果来决定我们是否应该停止搜索。通过我们的算法，称为动态模拟MCTS（DS-MCTS），我们可以将AlphaZero训练的NoGo代理的速度提高2.5倍，同时保持类似的胜率。在平均模拟次数相同的情况下，与原方案相比，该方法的胜率可达61%。
英文摘要：
Monte Carlo tree search (MCTS) has achieved state-of-the-art results in many domains such as Go and Atari games when combining with deep neural networks (DNNs). When more simulations are executed, MCTS can achieve higher performance but also requires enormous amounts of CPU and GPU resources. However, not all states require a long searching time to identify the best action that the agent can find. For example, in 19x19 Go and NoGo, we found that for more than half of the states, the best action predicted by DNN remains unchanged even after searching 2 minutes. This implies that a significant amount of resources can be saved if we are able to stop the searching earlier when we are confident with the current searching result. In this paper, we propose to achieve this goal by predicting the uncertainty of the current searching status and use the result to decide whether we should stop searching. With our algorithm, called Dynamic Simulation MCTS (DS-MCTS), we can speed up a NoGo agent trained by AlphaZero 2.5 times faster while maintaining a similar winning rate. Also, under the same average simulation count, our method can achieve a 61% winning rate against the original program.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
