arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1706.01077cs.AI
收藏

动力学部分已知的线性可解连续MDP的Actor-Critical

Actor-Critic for Linearly-Solvable Continuous MDP with Partially Known Dynamics

Toyota Central R & D Labs. Inc(丰田中央研发实验室公司);University of Georgia(佐治亚大学);Toyota Research Institute(丰田研究院);Toyota R & D(丰田研发)

Tomoki Nishi, Prashant Doshi, Michael R. James, Danil Prokhorov

摘要:

在许多机器人应用中,系统动力学的某些方面可以精确建模,而其他方面则难以获得或建模。我们提出了一种新的强化学习(RL)方法,连续的状态和动作空间,学习与系统的部分知识,没有积极的探索。它解决了线性可解马尔可夫决策过程(L-MDP),这是非常适合于连续的状态和动作空间,基于一个演员批评架构。与先前的用于L-MDP的RL方法和基于模型的路径积分方法相比,行动者-批评者学习不需要不受控制的动态模型,重要的是,不需要过渡噪声水平;然而,它需要知道问题的控制动态。我们评估我们的方法上的两个合成测试问题,和一个现实世界的问题,在模拟和使用真实的交通数据。我们的实验证明了改进的学习和策略性能。

英文摘要:

In many robotic applications, some aspects of the system dynamics can be modeled accurately while others are difficult to obtain or model. We present a novel reinforcement learning (RL) method for continuous state and action spaces that learns with partial knowledge of the system and without active exploration. It solves linearly-solvable Markov decision processes (L-MDPs), which are well suited for continuous state and action spaces, based on an actor-critic architecture. Compared to previous RL methods for L-MDPs and path integral methods which are model based, the actor-critic learning does not need a model of the uncontrolled dynamics and, importantly, transition noise levels; however, it requires knowing the control dynamics for the problem. We evaluate our method on two synthetic test problems, and one real-world problem in simulation and using real traffic data. Our experiments demonstrate improved learning and policy performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑