arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.05879cs.LGstat.ML
收藏

样本复杂度依赖间隙的马尔可夫决策过程的规划

Planning in Markov Decision Processes with Gap-Dependent Sample Complexity

Universitat Pompeu Fabra(庞培法布拉大学);CNRS(法国国家科学研究中心);ULille(里尔大学);Inria(法国国家信息与自动化研究所);Renault(雷诺公司);DeepMind(深度思维)

Anders Jonsson, Emilie Kaufmann, Pierre Ménard, Omar Darwiche Domingues, Edouard Leurent, Michal Valko

摘要:

我们提出了MDP-GapE,一个新的基于概率的蒙特卡罗树搜索算法规划在马尔可夫决策过程中,过渡有一个有限的支持。我们证明了MDP-GapE所需的生成模型的调用数量的上限,以确定具有高概率的接近最佳的动作。这个依赖于问题的样本复杂性结果表示在探索过程中访问的状态-动作对的次优差距。我们的实验表明,MDP-GapE在实践中也是有效的,与其他算法相比,样本复杂度的保证,在固定的置信度设置,这主要是理论上的。

英文摘要:

We propose MDP-GapE, a new trajectory-based Monte-Carlo Tree Search algorithm for planning in a Markov Decision Process in which transitions have a finite support. We prove an upper bound on the number of calls to the generative models needed for MDP-GapE to identify a near-optimal action with high probability. This problem-dependent sample complexity result is expressed in terms of the sub-optimality gaps of the state-action pairs that are visited during exploration. Our experiments reveal that MDP-GapE is also effective in practice, in contrast with other algorithms with sample complexity guarantees in the fixed-confidence setting, that are mostly theoretical.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑