单计数器MDP和随机对策终止值的逼近
Approximating the Termination Value of One-Counter MDPs and Stochastic Games
摘要:
单计数器MDP(OC-MDP)和单计数器简单随机游戏(OC-SSG)是在经典单计数器自动机(相当于具有1个字母堆栈字母表的下推自动机)的转换图上进行的单玩家和2玩家回合制零和随机游戏。这些游戏的分析和验证的一个关键目标是终止目标,其中玩家的目标是从给定的控制状态和给定的计数器值开始,最大化(分别最小化)命中计数器值0的概率。最近,我们研究了定性决策问题(“最优终止值= 1?“),并表明它们在P时间(分别在NP和coNP中)是可判定的。但是,定量决策和近似问题(“是最佳终止值?p”,或“近似在100以内的终止值”)更具挑战性。这部分是因为最优策略可能不存在,而且即使它们存在,它们也可能具有高度非平凡的结构。因此,即使这些定量终止问题中的任何一个是可计算的,它仍然是开放的。在本文中,我们表明,所有的定量逼近问题的终止值的OC-MDPs和OC-SSGs是可计算的。具体地说,给定一个OC-SSG,并且给定ε> 0,我们可以计算一个值v,它近似于OC-SSG终止博弈在加性误差ε内的值,此外,我们可以计算博弈中双方的ε-最优策略。在我们的证明中的一个关键因素是一个微妙的鞅,来自解决某些LP,我们可以与最大化的OC-MDP。Azuma不等式在这些鞅上的应用产生了一个可计算的“财富”界限,在这个界限上,“富人的策略”对于OC-MDP来说是ε-最优的。
英文摘要:
One-counter MDPs (OC-MDPs) and one-counter simple stochastic games (OC-SSGs) are 1-player, and 2-player turn-based zero-sum, stochastic games played on the transition graph of classic one-counter automata (equivalently, pushdown automata with a 1-letter stack alphabet). A key objective for the analysis and verification of these games is the termination objective, where the players aim to maximize (minimize, respectively) the probability of hitting counter value 0, starting at a given control state and given counter value. Recently, we studied qualitative decision problems ("is the optimal termination value = 1?") for OC-MDPs (and OC-SSGs) and showed them to be decidable in P-time (in NP and coNP, respectively). However, quantitative decision and approximation problems ("is the optimal termination value ? p", or "approximate the termination value within epsilon") are far more challenging. This is so in part because optimal strategies may not exist, and because even when they do exist they can have a highly non-trivial structure. It thus remained open even whether any of these quantitative termination problems are computable. In this paper we show that all quantitative approximation problems for the termination value for OC-MDPs and OC-SSGs are computable. Specifically, given a OC-SSG, and given epsilon > 0, we can compute a value v that approximates the value of the OC-SSG termination game within additive error epsilon, and furthermore we can compute epsilon-optimal strategies for both players in the game. A key ingredient in our proofs is a subtle martingale, derived from solving certain LPs that we can associate with a maximizing OC-MDP. An application of Azuma's inequality on these martingales yields a computable bound for the "wealth" at which a "rich person's strategy" becomes epsilon-optimal for OC-MDPs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。