arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1209.3617cs.GT
收藏

有限时域马尔可夫决策过程和简单随机对策的策略复杂性

Strategy complexity of finite-horizon Markov decision processes and simple stochastic games

Krishnendu Chatterjee, Rasmus Ibsen-Jensen

摘要:

马尔可夫决策过程(MDP)和简单随机博弈(SSG)为研究概率系统的许多重要问题提供了丰富的数学框架。具有有限时间目标的MDP和SSG是一个经典的研究问题,其目标是在给定的有限时间内最大化到达目标状态的概率。在这项工作中,我们考虑的策略复杂性有限的地平线MDPs和SSGs。我们发现,对于所有的$ε>0$,自然类的计数器为基础的策略需要在最$\log \log (\frac{1}ε) + n+1$内存状态,内存的大小$Ω(\log \log (\frac{1}ε) + n)$是必需的。因此,我们的边界是渐近最优的。然后,我们研究了最优策略的周期性,并给出了最优策略周期的一个次指数下界。

英文摘要:

Markov decision processes (MDPs) and simple stochastic games (SSGs) provide a rich mathematical framework to study many important problems related to probabilistic systems. MDPs and SSGs with finite-horizon objectives, where the goal is to maximize the probability to reach a target state in a given finite time, is a classical and well-studied problem. In this work we consider the strategy complexity of finite-horizon MDPs and SSGs. We show that for all $ε>0$, the natural class of counter-based strategies require at most $\log \log (\frac{1}ε) + n+1$ memory states, and memory of size $Ω(\log \log (\frac{1}ε) + n)$ is required. Thus our bounds are asymptotically optimal. We then study the periodic property of optimal strategies, and show a sub-exponential lower bound on the period for optimal strategies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑