In this paper, we develop a method to automatically generate a control policy for a dynamical system modeled as a Markov Decision Process (MDP). The control specification is given as a Linear Temporal Logic (LTL) formula over a set of propositions defined on the states of the MDP. We synthesize a control policy such that the MDP satisfies the given specification almost surely, if such a policy exists. In addition, we designate an "optimizing proposition" to be repeatedly satisfied, and we formulate a novel optimization criterion in terms of minimizing the expected cost in between satisfactions of this proposition. We propose a sufficient condition for a policy to be optimal, and develop a dynamic programming algorithm that synthesizes a policy that is optimal under some conditions, and sub-optimal otherwise. This problem is motivated by robotic applications requiring persistent tasks, such as environmental monitoring or data gathering, to be performed.
Automated Generation of MDPs Using Logic Programming and LLMs for Robotic Applications
Department of Information Engineering and Computer Science, University of Trento(信息工程与计算机科学系,特伦托大学);Department of Industrial Engineering, University of Trento(工业工程系,特伦托大学)
Planning with State Abstractions for Non-Markovian Task Specifications
Brown University(布朗大学)
通常,我们使用也可以跨越不同抽象级别的时态语言为机器人指定任务。示例命令"go to the kitchen before going to second floor“包含空间抽象,因为”floor“由单独的房间组成,这些房间也可以单独引用(例如“kitchen”)。还有一个时间顺序的事件,定义为“之前”一词。以前的工作已经使用线性时态逻辑(LTL)来解释时间语言(如“之前”),和抽象马尔可夫决策过程(AMDPs)解释层次抽象(如“厨房”和“二楼”),分别。为了同时处理这两种类型的命令,我们引入了抽象产品马尔可夫决策过程(AP-MDP),这是一种能够在不同抽象层次上表示非马尔可夫奖励函数的新方法。AP-MDP框架将LTL转换为相应的自动机,创建LTL规范和环境MDP的产品马尔可夫决策过程(MDP),并将问题分解为子问题,以实现具有抽象的有效规划。在超过95%的任务中,AP-MDP比解决LTL问题的非分层方法执行得更快,并且这个数字只会随着环境域的大小增加而增加。我们还提出了一个神经序列到序列模型训练语言命令翻译成LTL表达,和一个新的语料库的非马尔可夫语言命令跨越不同层次的抽象。我们测试我们的框架与收集的语言命令在无人机上,证明我们的方法使机器人能够有效地解决时间命令在不同的抽象层次。
Preference-Based Planning in Stochastic Environments: From Partially-Ordered Temporal Goals to Most Preferred Policies
Missouri State University(密苏里州立大学);University of Texas at Austin(德克萨斯大学奥斯汀分校);University of Florida(佛罗里达大学);Oden Institute for Computational Engineering and Sciences(奥登计算工程与科学研究所)