策略迭代非常适合优化PageRank
Policy Iteration is well suited to optimize PageRank
摘要：
在过去的50年里，关于求解马尔可夫决策过程（MDP）的策略迭代算法（PI）是具有指数复杂度还是（强）多项式复杂度的问题引起了人们的广泛关注。最近，Fearnley提出了一个例子，PI需要指数数量的迭代来收敛。尽管如此，人们已经观察到，Fearnley的例子留下了PI在许多特定情况下表现良好的可能性，例如在涉及固定折扣因子或仅限于确定性行为的问题中。在本文中，我们分析了一大类的MDP，我们认为PI是有效的，在这种情况下。这类问题是在优化马尔可夫链中特定节点的PageRank时得到的。它们是由几个实际应用所激发的。我们表明，添加自然约束，这个PageRank优化问题（PRO），使其等价于优化随机路径的长度的问题，这是一个广泛研究的家庭的MDP。最后，我们推测，PI运行在一个多项式的迭代次数时，适用于PRO。我们给出数值参数以及证明我们的猜想在一些特殊情况下的实际意义。
英文摘要：
The question of knowing whether the policy Iteration algorithm (PI) for solving Markov Decision Processes (MDPs) has exponential or (strongly) polynomial complexity has attracted much attention in the last 50 years. Recently, Fearnley proposed an example on which PI needs an exponential number of iterations to converge. Though, it has been observed that Fearnley's example leaves open the possibility that PI behaves well in many particular cases, such as in problems that involve a fixed discount factor, or that are restricted to deterministic actions. In this paper, we analyze a large class of MDPs and we argue that PI is efficient in that case. The problems in this class are obtained when optimizing the PageRank of a particular node in the Markov chain. They are motivated by several practical applications. We show that adding natural constraints to this PageRank Optimization problem (PRO) makes it equivalent to the problem of optimizing the length of a stochastic path, which is a widely studied family of MDPs. Finally, we conjecture that PI runs in a polynomial number of iterations when applied to PRO. We give numerical arguments as well as the proof of our conjecture in a number of particular cases of practical importance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
