概率Min/Max多项式方程的最大不动点与分支Markov决策过程的可达性
Greatest Fixed Points of Probabilistic Min/Max Polynomial Equations, and Reachability for Branching Markov Decision Processes
摘要：
给出了分支马尔可夫决策过程（BMDPs）定量（和定性）可达性分析的多项式时间算法。具体地说，给定BMDP和给定初始种群，其中控制器的目标是最大化（或最小化）最终到达包含期望的对象的群体的概率。（或不期望的）类型，我们给出了近似上确界（下确界）可达概率的算法，在期望的精度内，在BMDP的编码大小的时间多项式中以及在log（1/log）中。此外，我们给P-时间算法计算ε-最优策略的最大化和最小化的可达性概率。我们也给P-时间算法的所有相关的定性分析问题，即：决定是否最佳（上确界或下确界）的可达概率为0或1。在本文之前，BMDPs的最优可达概率的近似甚至不知道是可判定的。我们的算法利用以下基本事实：我们表明，对于任何BMDP，其最大（最小）不可达概率由相应的单调max（min）概率多项式方程组（max/min-PPS），x=P（x）的最大不动点（GFP）解g* 在[0，1]^n中给出，这是具有不可达目标的BMDP的Bellman最优性方程。我们展示了如何在P时间内计算最大/最小PPS的GFP到所需的精度。我们还研究了更一般的分支简单随机游戏（BSSGs）（非）可达性目标。我们证明：（1）这些游戏的价值由相应的max-minPPS的GFP捕获;（2）近似值的定量问题在TFNP中;以及（3）与值相关的定性问题都可以在P时间内解决。
英文摘要：
We give polynomial time algorithms for quantitative (and qualitative) reachability analysis for Branching Markov Decision Processes (BMDPs). Specifically, given a BMDP, and given an initial population, where the objective of the controller is to maximize (or minimize) the probability of eventually reaching a population that contains an object of a desired (or undesired) type, we give algorithms for approximating the supremum (infimum) reachability probability, within desired precision epsilon > 0, in time polynomial in the encoding size of the BMDP and in log(1/epsilon). We furthermore give P-time algorithms for computing epsilon-optimal strategies for both maximization and minimization of reachability probabilities. We also give P-time algorithms for all associated qualitative analysis problems, namely: deciding whether the optimal (supremum or infimum) reachability probabilities are 0 or 1. Prior to this paper, approximation of optimal reachability probabilities for BMDPs was not even known to be decidable. Our algorithms exploit the following basic fact: we show that for any BMDP, its maximum (minimum) non-reachability probabilities are given by the greatest fixed point (GFP) solution g* in [0,1]^n of a corresponding monotone max (min) Probabilistic Polynomial System of equations (max/min-PPS), x=P(x), which are the Bellman optimality equations for a BMDP with non-reachability objectives. We show how to compute the GFP of max/min PPSs to desired precision in P-time. We also study more general Branching Simple Stochastic Games (BSSGs) with (non-)reachability objectives. We show that: (1) the value of these games is captured by the GFP of a corresponding max-minPPS; (2) the quantitative problem of approximating the value is in TFNP; and (3) the qualitative problems associated with the value are all solvable in P-time.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
