arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1202.2185cs.ROcs.SYeess.SYmath.OC
收藏

使用Actor-Critic方法的时序逻辑运动控制

Temporal Logic Motion Control using Actor-Critic Methods

United Technologies Research Center(联合技术研究中心);Boston University(波士顿大学)

Xu Chu Ding, Jing Wang, Morteza Lahijanian, Ioannis Ch. Paschalidis, Calin A. Belta

AI总结:

针对大型分区环境中基于时序逻辑规范的控制问题,提出一种基于最小二乘时序差分学习的Actor-Critic近似动态规划框架,通过优化随机控制策略参数实现近似最优策略。

摘要:

本文考虑从以时序逻辑语句形式给出的规范部署机器人的问题,该规范涉及大型分区环境中区域满足的某些属性。我们假设机器人具有噪声传感器和执行器,并将其在环境区域中的运动建模为马尔可夫决策过程(MDP)。机器人控制问题变为寻找在MDP上最大化满足时序逻辑任务概率的控制策略。对于大型环境,获取每个状态-动作对的转移概率以及求解最优策略所需的优化问题通常计算上不可行。为解决这些问题,我们提出了一种基于最小二乘时序差分学习方法的Actor-Critic类型近似动态规划框架。该框架在机器人的样本路径上运行,并针对少量参数优化随机控制策略。转移概率仅在需要时获取。硬件在环仿真证实,参数的收敛转化为近似最优策略。

英文摘要:

In this paper, we consider the problem of deploying a robot from a specification given as a temporal logic statement about some properties satisfied by the regions of a large, partitioned environment. We assume that the robot has noisy sensors and actuators and model its motion through the regions of the environment as a Markov Decision Process (MDP). The robot control problem becomes finding the control policy maximizing the probability of satisfying the temporal logic task on the MDP. For a large environment, obtaining transition probabilities for each state-action pair, as well as solving the necessary optimization problem for the optimal policy are usually not computationally feasible. To address these issues, we propose an approximate dynamic programming framework based on a least-square temporal difference learning method of the actor-critic type. This framework operates on sample paths of the robot and optimizes a randomized control policy with respect to a small set of parameters. The transition probabilities are obtained only when needed. Hardware-in-the-loop simulations confirm that convergence of the parameters translates to an approximately optimal policy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑