arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1206.3266cs.AI
收藏

MDP的分块线性规划逼近

Partitioned Linear Programming Approximations for MDPs

Intel Research(英特尔研究院);University of Pittsburgh(匹兹堡大学)

Branislav Kveton, Milos Hauskrecht

摘要:

近似线性规划(ALP)是求解大因子马尔可夫决策过程(MDP)的一种有效方法。该方法的主要思想是用一组基函数逼近最优价值函数,并通过线性规划(LP)优化其权重。本文提出了一种新的ALP近似。与标准ALP公式相比,我们将约束空间分解为一组低维空间。这种结构允许有效地求解新的LP。特别地,LP的约束可以以紧凑的形式满足,而不对ALP约束的树宽具有指数依赖性。我们研究所提出的方法的实践和理论方面。此外,我们证明了它的放大潜力的MDP与超过2^100个国家。

英文摘要:

Approximate linear programming (ALP) is an efficient approach to solving large factored Markov decision processes (MDPs). The main idea of the method is to approximate the optimal value function by a set of basis functions and optimize their weights by linear programming (LP). This paper proposes a new ALP approximation. Comparing to the standard ALP formulation, we decompose the constraint space into a set of low-dimensional spaces. This structure allows for solving the new LP efficiently. In particular, the constraints of the LP can be satisfied in a compact form without an exponential dependence on the treewidth of ALP constraints. We study both practical and theoretical aspects of the proposed approach. Moreover, we demonstrate its scale-up potential on an MDP with more than 2^100 states.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑