arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1912.11223cs.LOmath.OC
收藏

基于场景的不确定MDP验证

Scenario-Based Verification of Uncertain MDPs

Murat Cubuktepe, Nils Jansen, Sebastian Junges, Joost-Pieter Katoen, Ufuk Topcu

摘要:

我们考虑马尔可夫决策过程(MDP)的转移概率和奖励属于一个不确定性集参数化的随机变量的集合。这些随机参数的概率分布是未知的。问题是要计算的概率,以满足任何MDP,对应于从这些未知的分布样本的时间逻辑规范。一般来说,这个问题是不可判定的,我们求助于所谓的场景优化技术。基于有限数量的样本的不确定参数,其中每一个都导致MDP,所提出的方法估计满足规格的概率,通过求解有限维凸优化问题。在此估计上获得高置信度所需的样本数与状态数和随机参数数无关。在大量基准测试上的实验表明,几千个样本就足以以高概率获得高质量的置信区间。

英文摘要:

We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are unknown. The problem is to compute the probability to satisfy a temporal logic specification within any MDP that corresponds to a sample from these unknown distributions. In general, this problem is undecidable, and we resort to techniques from so-called scenario optimization. Based on a finite number of samples of the uncertain parameters, each of which induces an MDP, the proposed method estimates the probability of satisfying the specification by solving a finite-dimensional convex optimization problem. The number of samples required to obtain a high confidence on this estimate is independent from the number of states and the number of random parameters. Experiments on a large set of benchmarks show that a few thousand samples suffice to obtain high-quality confidence bounds with a high probability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑