arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1906.12239cs.LGstat.ML
收藏

基于L* 的马尔可夫决策过程学习(扩展版)

L*-Based Learning of Markov Decision Processes (Extended Version)

Graz University of Technology(格拉茨工业大学);Aalborg University(奥尔堡大学)

Martin Tappler, Bernhard K. Aichernig, Giovanni Bacci, Maria Eichlseder, Kim G. Larsen

摘要:

自动机学习技术从测试观察自动生成系统模型。这些技术通常分为两类:被动和主动。被动学习使用预定的数据集,例如,系统日志。相比之下,主动学习主动地查询学习中的系统,这被认为是更有效的。一个有影响力的主动学习技术是Angluin的L* 算法的正则语言,启发了几个概括从DFA到其他自动机为基础的建模形式主义。在这项工作中,我们研究了基于L* 的学习确定性马尔可夫决策过程,首先假设一个理想的设置与完美的信息。然后,我们放松了这一假设,并提出了一种新的学习算法,收集信息的采样系统的痕迹,通过测试。我们的采样为基础的算法的实施实验表明,它实现了更好的准确性比国家的最先进的被动学习技术与相同数量的测试数据。不同于现有的学习算法与预定义的状态,我们的算法学习完整的模型结构,包括状态。

英文摘要:

Automata learning techniques automatically generate system models from test observations. These techniques usually fall into two categories: passive and active. Passive learning uses a predetermined data set, e.g., system logs. In contrast, active learning actively queries the system under learning, which is considered more efficient. An influential active learning technique is Angluin's L* algorithm for regular languages which inspired several generalisations from DFAs to other automata-based modelling formalisms. In this work, we study L*-based learning of deterministic Markov decision processes, first assuming an ideal setting with perfect information. Then, we relax this assumption and present a novel learning algorithm that collects information by sampling system traces via testing. Experiments with the implementation of our sampling-based algorithm suggest that it achieves better accuracy than state-of-the-art passive learning techniques with the same amount of test data. Unlike existing learning algorithms with predefined states, our algorithm learns the complete model structure including the states.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑