可测蒙特卡罗搜索误差界
Measurable Monte Carlo Search Error Bounds
摘要：
蒙特卡洛规划器通常可以返回次优的行动，即使他们保证在无限样本的限制下收敛。已知的渐近后悔界限不提供任何方法来衡量信心的建议行动在搜索结束。在这项工作中，我们证明了非平稳土匪和马尔可夫决策过程的Monte Carlo估计的次最优性的界限。这些界限可以在搜索结束时直接计算，并且不需要了解真实的动作值。所提出的界限举行一般蒙特卡罗求解器满足温和的收敛条件。我们经验测试的紧密性的界限，通过实验上的多臂土匪和离散马尔可夫决策过程的一个简单的求解器和蒙特卡罗树搜索。
英文摘要：
Monte Carlo planners can often return sub-optimal actions, even if they are guaranteed to converge in the limit of infinite samples. Known asymptotic regret bounds do not provide any way to measure confidence of a recommended action at the conclusion of search. In this work, we prove bounds on the sub-optimality of Monte Carlo estimates for non-stationary bandits and Markov decision processes. These bounds can be directly computed at the conclusion of the search and do not require knowledge of the true action-value. The presented bound holds for general Monte Carlo solvers meeting mild convergence conditions. We empirically test the tightness of the bounds through experiments on a multi-armed bandit and a discrete Markov decision process for both a simple solver and Monte Carlo tree search.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
