带Pareto缓存的组合式值迭代
Compositional Value Iteration with Pareto Caching
AI总结:
针对MDP验证效率问题,提出带Pareto缓存的组合式值迭代框架,通过复用组件验证结果提升效率,设计两种停止准则保障可靠性,实验验证了其有效性。
摘要:
马尔可夫决策过程(MDP)验证的事实标准方法基于值迭代(VI)。我们提出组合式值迭代(compositional VI),这是一种用于组合式MDP模型检测的框架,在保持可靠性的同时提升效率。具体而言,组合式MDP自然产生于单个组件的组合,其结构可通过弦图等方式表达。为提升效率,我们发现组合式值迭代会重复验证单个组件,因此提出名为Pareto缓存的技术,即使针对此前未见过的查询也能复用验证结果。为保证可靠性,我们提出两种停止准则:一种推广了乐观值迭代范式,另一种将Pareto缓存与最新的基准算法结合使用。我们的实验评估展现了该新算法及其变体的应用前景,同时指出了未来工作的挑战。
英文摘要:
The de-facto standard approach in MDP verification is based on value iteration (VI). We propose compositional VI, a framework for model checking compositional MDPs, that addresses efficiency while maintaining soundness. Concretely, compositional MDPs naturally arise from the combination of individual components, and their structure can be expressed using, e.g., string diagrams. Towards efficiency, we observe that compositional VI repeatedly verifies individual components. We propose a technique called Pareto caching that allows to reuse verification results, even for previously unseen queries. Towards soundness, we present two stopping criteria: one generalizes the optimistic value iteration paradigm and the other uses Pareto caches in conjunction with recent baseline algorithms. Our experimental evaluations shows the promise of the novel algorithm and its variations, and identifies challenges for future work.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。