arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1402.2967cs.LO
收藏

基于学习算法的马尔可夫决策过程验证

Verification of Markov Decision Processes using Learning Algorithms

Tomáš Brázdil, Krishnendu Chatterjee, Martin Chmelík, Vojtěch Forejt, Jan Křetínský, Marta Kwiatkowska, David Parker, Mateusz Ujma

摘要:

我们提出了一个通用的框架,应用机器学习算法的马尔可夫决策过程(MDP)的验证。这些技术的主要目标是通过避免状态空间的穷举探索来提高性能。我们的框架侧重于概率可达性,这是一个核心属性验证,并说明通过两个不同的实例。第一个假设,MDP的全部知识是可用的,并执行一个数学驱动的部分探索的模型,产生精确的下限和上限所需的概率。第二个处理的情况下,我们可能只采样的MDP,并产生概率的保证,再次在上下界,这提供了有效的停止标准的近似。后者是第一次扩展的统计模型检查的无界属性在MDPs。与其他相关的方法相比,我们不限制我们的注意力时间有界(有限的地平线)或折扣属性,也不假设任何特定的属性的MDP。我们还展示了我们的技术如何扩展到LTL目标。我们目前的实验结果显示,我们的框架上的几个例子的性能。

英文摘要:

We present a general framework for applying machine-learning algorithms to the verification of Markov decision processes (MDPs). The primary goal of these techniques is to improve performance by avoiding an exhaustive exploration of the state space. Our framework focuses on probabilistic reachability, which is a core property for verification, and is illustrated through two distinct instantiations. The first assumes that full knowledge of the MDP is available, and performs a heuristic-driven partial exploration of the model, yielding precise lower and upper bounds on the required probability. The second tackles the case where we may only sample the MDP, and yields probabilistic guarantees, again in terms of both the lower and upper bounds, which provides efficient stopping criteria for the approximation. The latter is the first extension of statistical model-checking for unbounded properties in MDPs. In contrast with other related approaches, we do not restrict our attention to time-bounded (finite-horizon) or discounted properties, nor assume any particular properties of the MDP. We also show how our techniques extend to LTL objectives. We present experimental results showing the performance of our framework on several examples.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑