arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1103.4342cs.ROcs.SYeess.SYmath.OC
收藏

时序逻辑约束下的MDP最优控制

MDP Optimal Control under Temporal Logic Constraints

Boston University(波士顿大学);University of Waterloo(滑铁卢大学)

Xu Chu Ding, Stephen L. Smith, Calin Belta, Daniela Rus

AI总结:

针对马尔可夫决策过程(MDP),提出一种在给定线性时序逻辑(LTL)规范下自动生成控制策略的方法,并引入优化命题以最小化期望成本,通过动态规划算法合成最优或次优策略。

摘要:

在本文中,我们开发了一种方法,用于自动生成以马尔可夫决策过程(MDP)建模的动态系统的控制策略。控制规范以线性时序逻辑(LTL)公式给出,该公式基于定义在MDP状态上的一组命题。我们合成一个控制策略,使得MDP几乎必然满足给定规范(如果这样的策略存在)。此外,我们指定一个“优化命题”以重复满足,并制定了一个新的优化准则,即最小化该命题满足之间的期望成本。我们提出了策略最优的充分条件,并开发了一种动态规划算法,该算法在某些条件下合成最优策略,否则合成次优策略。此问题源于需要执行持久性任务的机器人应用,例如环境监测或数据收集。

英文摘要:

In this paper, we develop a method to automatically generate a control policy for a dynamical system modeled as a Markov Decision Process (MDP). The control specification is given as a Linear Temporal Logic (LTL) formula over a set of propositions defined on the states of the MDP. We synthesize a control policy such that the MDP satisfies the given specification almost surely, if such a policy exists. In addition, we designate an "optimizing proposition" to be repeatedly satisfied, and we formulate a novel optimization criterion in terms of minimizing the expected cost in between satisfactions of this proposition. We propose a sufficient condition for a policy to be optimal, and develop a dynamic programming algorithm that synthesizes a policy that is optimal under some conditions, and sub-optimal otherwise. This problem is motivated by robotic applications requiring persistent tasks, such as environmental monitoring or data gathering, to be performed.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑