arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1206.3233cs.AI
收藏

通过自动构造抽象加速马尔可夫决策过程中的规划

Speeding Up Planning in Markov Decision Processes via Automatically Constructed Abstractions

University of Alberta(阿尔伯塔大学)

Alejandro Isaza, Csaba Szepesvari, Vadim Bulitko, Russell Greiner

摘要:

在本文中,我们考虑规划随机最短路径(SSP)问题,马尔可夫决策问题(MDP)的一个子类。我们专注于中等规模的问题,其状态空间可以完全枚举。这个问题有许多重要的应用,如导航和规划下的不确定性。我们提出了一种新的方法来构建一个多层次的层次结构,逐步简单的抽象的原始问题。一旦计算完成,层次结构就可以用来加速规划,首先找到最抽象级别的策略,然后递归地将其细化为原始问题的解决方案。这种方法是完全自动化的,并提供了两个数量级的速度比最先进的MDP求解器的样本问题,同时返回接近最佳的解决方案。我们还证明了理论界的解决方案的最优性损失所造成的使用抽象。

英文摘要:

In this paper, we consider planning in stochastic shortest path (SSP) problems, a subclass of Markov Decision Problems (MDP). We focus on medium-size problems whose state space can be fully enumerated. This problem has numerous important applications, such as navigation and planning under uncertainty. We propose a new approach for constructing a multi-level hierarchy of progressively simpler abstractions of the original problem. Once computed, the hierarchy can be used to speed up planning by first finding a policy for the most abstract level and then recursively refining it into a solution to the original problem. This approach is fully automated and delivers a speed-up of two orders of magnitude over a state-of-the-art MDP solver on sample problems while returning near-optimal solutions. We also prove theoretical bounds on the loss of solution optimality resulting from the use of abstractions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑