arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1609.07409eess.SYcs.SY
收藏

Q学习用于满足信号时间逻辑规范的鲁棒性

Q-Learning for Robust Satisfaction of Signal Temporal Logic Specifications

Massachusetts Institute of Technology(麻省理工学院);Georgia Institute of Technology(佐治亚理工学院);Stanford University(斯坦福大学);Boston University(波士顿大学)

Derya Aksaray, Austin Jones, Zhaodan Kong, Mac Schwager, Calin Belta

AI总结:

本文研究了在未知随机动态下通过Q学习满足信号时间逻辑规范的问题,提出通过最大化满足概率和鲁棒度度量来解决合成问题,并通过仿真验证了方法性能。

摘要:

本文针对在未知随机动态下学习满足信号时间逻辑(STL)规范的最优策略问题,将系统建模为马尔可夫决策过程,其中状态表示连续空间的划分,转移概率未知。我们提出了两个合成问题,通过最大化满足概率和期望鲁棒度来强制满足所需的STL规范。讨论指出Q学习不能直接应用于这些问题,因为基于STL的定量语义,满足概率和期望鲁棒度度量不在Q学习的标准目标形式中。为解决此问题,我们提出了一种可由Q学习解决的STL合成问题近似方法,并推导了通过近似方法获得的策略的性能界限。通过仿真验证了所提方法的性能。

英文摘要:

This paper addresses the problem of learning optimal policies for satisfying signal temporal logic (STL) specifications by agents with unknown stochastic dynamics. The system is modeled as a Markov decision process, in which the states represent partitions of a continuous space and the transition probabilities are unknown. We formulate two synthesis problems where the desired STL specification is enforced by maximizing the probability of satisfaction, and the expected robustness degree, that is, a measure quantifying the quality of satisfaction. We discuss that Q-learning is not directly applicable to these problems because, based on the quantitative semantics of STL, the probability of satisfaction and expected robustness degree are not in the standard objective form of Q-learning. To resolve this issue, we propose an approximation of STL synthesis problems that can be solved via Q-learning, and we derive some performance bounds for the policies obtained by the approximate approach. The performance of the proposed method is demonstrated via simulations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑