arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1301.6718cs.AI
收藏

论政策迭代的复杂性

On the Complexity of Policy Iteration

AT&T Labs-Research(AT&T实验室-研究部);Tel-Aviv University(特拉维夫大学)

Yishay Mansour, Satinder Singh

摘要:

在不确定或随机域中的决策问题通常被表述为马尔可夫决策过程(MDP)。策略迭代(PI)是一种流行的策略空间搜索算法,其大小与状态数成指数关系。我们感兴趣的范围内的复杂性PI不依赖于折扣因子的值。在本文中,我们证明了第一个这样的非平凡的,最坏的情况下,上界PI收敛到最优策略所需的迭代次数。我们的分析还揭示了PI通过政策空间取得进展的方式。

英文摘要:

Decision-making problems in uncertain or stochastic domains are often formulated as Markov decision processes (MDPs). Policy iteration (PI) is a popular algorithm for searching over policy-space, the size of which is exponential in the number of states. We are interested in bounds on the complexity of PI that do not depend on the value of the discount factor. In this paper we prove the first such non-trivial, worst-case, upper bounds on the number of iterations required by PI to converge to the optimal policy. Our analysis also sheds new light on the manner in which PI progresses through the space of policies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑