马尔可夫决策过程的鲁棒同步
Robust Synchronization in Markov Decision Processes
摘要：
我们认为同步性能的马尔可夫决策过程（MDP），视为发电机的概率分布序列的状态。一个概率分布是p-同步的，如果在某些状态下概率质量至少是p，而一个概率分布序列是弱p-同步的，或者是强p-同步的，如果序列中分别有无穷多个或除了无穷多个分布都是p-同步的。对于每种同步模式，MDP可以是（i）如果存在产生1-同步序列的策略，则确定获胜;（ii）如果存在产生对于所有ε > 0而言是（1-ε）-同步序列的序列的策略，则几乎确定获胜;（iii）如果对于所有ε > 0而言，存在产生（1-ε）-同步序列的策略，则有限确定获胜。对于每一种同步和获胜模式，我们考虑了MDP是否获胜的问题，并建立了问题的上、下复杂度界，以及获胜策略的最优内存需求：（a）对于所有获胜模式，我们证明了问题对于弱同步是PSPACE完全的，对于强同步是PTIME完全的;（b）我们证明了对于弱同步，指数记忆是足够的，并且可能是必赢的，无限记忆是必赢的;对于强同步，线性记忆是足够的，并且可能在所有模式下都是必赢的;（3）我们给出了一个鲁棒性结果，即几乎确定和极限确定获胜模式对于弱同步和强同步都是一致的。
英文摘要：
We consider synchronizing properties of Markov decision processes (MDP), viewed as generators of sequences of probability distributions over states. A probability distribution is p-synchronizing if the probability mass is at least p in some state, and a sequence of probability distributions is weakly p-synchronizing, or strongly p-synchronizing if respectively infinitely many, or all but finitely many distributions in the sequence are p-synchronizing. For each synchronizing mode, an MDP can be (i) sure winning if there is a strategy that produces a 1-synchronizing sequence; (ii) almost-sure winning if there is a strategy that produces a sequence that is, for all ε > 0, a (1-ε)-synchronizing sequence; (iii) limit-sure winning if for all ε > 0, there is a strategy that produces a (1-ε)-synchronizing sequence. For each synchronizing and winning mode, we consider the problem of deciding whether an MDP is winning, and we establish matching upper and lower complexity bounds of the problems, as well as the optimal memory requirement for winning strategies: (a) for all winning modes, we show that the problems are PSPACE-complete for weakly synchronizing, and PTIME-complete for strongly synchronizing; (b) we show that for weakly synchronizing, exponential memory is sufficient and may be necessary for sure winning, and infinite memory is necessary for almost-sure winning; for strongly synchronizing, linear-size memory is sufficient and may be necessary in all modes; (c) we show a robustness result that the almost-sure and limit-sure winning modes coincide for both weakly and strongly synchronizing.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
