arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1409.5486eess.SYcs.SY
收藏

基于学习的方法用于线性时序逻辑规范的马尔可夫决策过程控制综合

A Learning Based Approach to Control Synthesis of Markov Decision Processes for Linear Temporal Logic Specifications

University of California, Berkeley(加州大学伯克利分校)

Dorsa Sadigh, Eric S. Kim, Samuel Coogan, S. Shankar Sastry, Sanjit A. Seshia

AI总结:

本文提出通过构建产品马尔可夫决策过程来合成满足线性时序逻辑规范的控制器,利用学习理论技术处理未知转移概率的情况,并通过交通控制案例验证方法的有效性。

摘要:

我们提出了一种方法,用于合成马尔可夫决策过程(MDP)的控制策略,使得MDP产生的轨迹满足线性时序逻辑(LTL)属性。我们构建了一个产品MDP,其中包含由所需LTL属性生成的确定性Rabin自动机。产品MDP的奖励函数从Rabin自动机的接受条件定义。这种构造允许我们将学习理论技术应用于LTL规范的综合问题,即使在事先不知道转移概率的情况下。我们证明,如果存在满足LTL属性的策略,我们的方法可以保证以概率一找到这样的控制器,并通过交通控制案例研究证明,即使无法以概率一满足LTL属性,我们的方法也能产生合理的控制策略。

英文摘要:

We propose to synthesize a control policy for a Markov decision process (MDP) such that the resulting traces of the MDP satisfy a linear temporal logic (LTL) property. We construct a product MDP that incorporates a deterministic Rabin automaton generated from the desired LTL property. The reward function of the product MDP is defined from the acceptance condition of the Rabin automaton. This construction allows us to apply techniques from learning theory to the problem of synthesis for LTL specifications even when the transition probabilities are not known a priori. We prove that our method is guaranteed to find a controller that satisfies the LTL property with probability one if such a policy exists, and we suggest empirically with a case study in traffic control that our method produces reasonable control strategies even when the LTL property cannot be satisfied with probability one.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑