分支Markov决策过程的多项式时间算法和概率最小（最大）多项式Bellman方程
Polynomial Time Algorithms for Branching Markov Decision Processes and Probabilistic Min(Max) Polynomial Bellman Equations
摘要：
我们表明，可以近似的最小不动点的解决方案的多变量系统的单调概率最大（最小）多项式方程，称为maxPPS（和minPPS，分别），在时间多项式的编码大小的方程组和log（1/1），其中1/2> 0是所需的附加误差界的解决方案。(The计算模型是标准的图灵机模型。）我们建立了这个结果，使用牛顿的方法，适用于maxPPS和minPPS的推广，即使基本功能是分段可微的。这概括了我们最近的工作，提供了一个P-时间算法的纯概率PPS。这些方程形成了几类重要的无限状态马尔可夫决策过程（MDP）的贝尔曼最优性方程。因此，作为一个推论，我们得到的第一个多项式时间算法计算任意所需的精度内的最佳值向量的几个类的无限状态MDP出现的经典的扩展，大量研究，纯随机过程。这些问题包括最大化和最小化多类型分支MDP，随机上下文无关MDP和1-出口递归MDP的终止（灭绝）概率的问题。此外，我们还表明，我们可以计算在P-时间的最大化和最小化分支，上下文无关，和1-出口递归MDPs，对于任何给定的期望的ε> 0的ε-最优策略。尽管在这种情况下，实际上计算最优策略是Sqrt-Sum-hard和PosSLP-hard。我们还推导出，作为一个简单的后果，这些结果，FNP上界的复杂性计算的值（在任意所需的精度）的分支简单随机游戏（BSSGs）。
英文摘要：
We show that one can approximate the least fixed point solution for a multivariate system of monotone probabilistic max(min) polynomial equations, referred to as maxPPSs (and minPPSs, respectively), in time polynomial in both the encoding size of the system of equations and in log(1/epsilon), where epsilon > 0 is the desired additive error bound of the solution. (The model of computation is the standard Turing machine model.) We establish this result using a generalization of Newton's method which applies to maxPPSs and minPPSs, even though the underlying functions are only piecewise-differentiable. This generalizes our recent work which provided a P-time algorithm for purely probabilistic PPSs. These equations form the Bellman optimality equations for several important classes of infinite-state Markov Decision Processes (MDPs). Thus, as a corollary, we obtain the first polynomial time algorithms for computing to within arbitrary desired precision the optimal value vector for several classes of infinite-state MDPs which arise as extensions of classic, and heavily studied, purely stochastic processes. These include both the problem of maximizing and mininizing the termination (extinction) probability of multi-type branching MDPs, stochastic context-free MDPs, and 1-exit Recursive MDPs. Furthermore, we also show that we can compute in P-time an epsilon-optimal policy for both maximizing and minimizing branching, context-free, and 1-exit-Recursive MDPs, for any given desired epsilon > 0. This is despite the fact that actually computing optimal strategies is Sqrt-Sum-hard and PosSLP-hard in this setting. We also derive, as an easy consequence of these results, an FNP upper bound on the complexity of computing the value (within arbitrary desired precision) of branching simple stochastic games (BSSGs).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
