存在转移不确定性时的蒙特卡洛树搜索
Monte Carlo Tree Search in the Presence of Transition Uncertainty
AI总结：
针对环境模型不完美导致标准MCTS性能下降的问题，提出不确定性自适应MCTS（UA-MCTS），通过估计转移不确定性并引导搜索，改进四个阶段，在损坏赌博机中取得更紧遗憾界，实验验证其显著提升性能。
摘要：
蒙特卡洛树搜索（MCTS）是一种广泛用于决策的基于搜索的框架。传统上，它应用于环境完美模拟模型可用的领域。我们研究并改进了在环境模型给定但不完美的情况下的MCTS。我们表明，模型与实际环境之间的差异可能导致标准MCTS的性能显著下降。因此，我们开发了不确定性自适应MCTS（UA-MCTS），这是MCTS框架内一种更鲁棒的算法。我们估计给定模型中的转移不确定性，并将搜索引导向状态空间中更确定的转移。我们修改了所有四个MCTS阶段，通过考虑这些估计来改善搜索行为。我们证明，在损坏的赌博机情况下，将不确定性信息添加到UCB中以适应，比标准UCB产生更紧的遗憾界。在实验上，我们在MinAtar测试套件的确定性域上评估了UA-MCTS及其各个组件。我们的结果表明，在存在模型转移错误的情况下，UA-MCTS显著改善了MCTS。
英文摘要：
Monte Carlo Tree Search (MCTS) is an immensely popular search-based framework used for decision making. It is traditionally applied to domains where a perfect simulation model of the environment is available. We study and improve MCTS in the context where the environment model is given but imperfect. We show that the discrepancy between the model and the actual environment can lead to significant performance degradation with standard MCTS. We therefore develop Uncertainty Adapted MCTS (UA-MCTS), a more robust algorithm within the MCTS framework. We estimate the transition uncertainty in the given model, and direct the search towards more certain transitions in the state space. We modify all four MCTS phases to improve the search behavior by considering these estimates. We prove, in the corrupted bandit case, that adding uncertainty information to adapt UCB leads to tighter regret bound than standard UCB. Empirically, we evaluate UA-MCTS and its individual components on the deterministic domains from the MinAtar test suite. Our results demonstrate that UA-MCTS strongly improves MCTS in the presence of model transition errors.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
