1-2-3-Go! 基于决策树学习与泛化的参数化马尔可夫决策过程策略合成
1-2-3-Go! Policy Synthesis for Parameterized Markov Decision Processes via Decision-Tree Learning and Generalization
AI总结：
针对参数化MDP的状态空间爆炸问题，提出利用决策树学习将小实例的最优策略泛化至大实例，无需显式探索大型模型状态空间，在远超现有工具处理规模的模型上表现良好。
摘要：
尽管概率模型检查取得了进展，但验证方法的可扩展性仍然有限。特别是，即使使用适中的值对参数化马尔可夫决策过程（MDPs）进行实例化，状态空间通常也会变得极其庞大。为如此庞大的MDP合成策略超出了现有工具的能力范围。我们提出了一种基于学习的方法，以为此类庞大的MDP获取合理的策略。其核心思想是利用决策树学习，将通过模型检查小实例获得的最优策略泛化到更大的实例上。因此，我们的方法避免了对大型模型进行显式的状态空间探索，为状态空间爆炸问题提供了实用的解决方案。我们通过在定量验证基准集的相关模型上进行广泛实验，展示了我们方法的有效性。实验结果表明，即使模型大小比最先进分析工具的处理能力高出数个数量级，我们的策略依然表现良好。
英文摘要：
Despite the advances in probabilistic model checking, the scalability of the verification methods remains limited. In particular, the state space often becomes extremely large when instantiating parameterized Markov decision processes (MDPs) even with moderate values. Synthesizing policies for such \emph{huge} MDPs is beyond the reach of available tools. We propose a learning-based approach to obtain a reasonable policy for such huge MDPs. The idea is to generalize optimal policies obtained by model-checking small instances to larger ones using decision-tree learning. Consequently, our method bypasses the need for explicit state-space exploration of large models, providing a practical solution to the state-space explosion problem. We demonstrate the efficacy of our approach by performing extensive experimentation on the relevant models from the quantitative verification benchmark set. The experimental results indicate that our policies perform well, even when the size of the model is orders of magnitude beyond the reach of state-of-the-art analysis tools.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
