选择计算:理论与应用
Selecting Computations: Theory and Applications
摘要:
序列决策问题通常可以通过模拟未来可能的行动序列来近似求解。已经开发了{ \em Metalevel}决策过程,用于基于估计将由任何特定模拟产生的决策质量的预期改进来选择{ \em which}动作序列进行模拟;一个例子是最近在围棋游戏中使用强盗算法来控制蒙特卡罗树搜索的工作。在本文中,我们开发的贝叶斯{ \em选择问题}的统计框架中的元级决策的理论基础,认为(如其他人所做的),这是更合适的比强盗框架。我们得出了一些基本的结果适用于蒙特卡洛选择问题,包括第一有限的抽样范围在某些情况下的最优政策,我们还提供了一个简单的反例直观的猜想,最优政策将必然达到在所有情况下的决定。然后,我们推导出启发式近似在贝叶斯和分布的自由设置,并证明其优越性,以土匪为基础的一杆决策问题,并在围棋。
英文摘要:
Sequential decision problems are often approximately solvable by simulating possible future action sequences. {\em Metalevel} decision procedures have been developed for selecting {\em which} action sequences to simulate, based on estimating the expected improvement in decision quality that would result from any particular simulation; an example is the recent work on using bandit algorithms to control Monte Carlo tree search in the game of Go. In this paper we develop a theoretical basis for metalevel decisions in the statistical framework of Bayesian {\em selection problems}, arguing (as others have done) that this is more appropriate than the bandit framework. We derive a number of basic results applicable to Monte Carlo selection problems, including the first finite sampling bounds for optimal policies in certain cases; we also provide a simple counterexample to the intuitive conjecture that an optimal policy will necessarily reach a decision in all cases. We then derive heuristic approximations in both Bayesian and distribution-free settings and demonstrate their superiority to bandit-based heuristics in one-shot decision problems and in Go.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。