arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1709.07932cs.RO
收藏

通过中继神经网络扩展运动技能

Expanding Motor Skills through Relay Neural Networks

Georgia Institute of Technology(佐治亚理工学院);Google Brain(谷歌大脑)

Visak C. V. Kumar, Sehoon Ha, C. Karen Liu

摘要:

虽然深度强化学习的最新进展在学习运动技能方面取得了令人印象深刻的成果,但许多经过训练的策略只能在有限的初始状态集合中使用。我们提出了一种技术,将复杂的机器人任务分解为简单的子任务,并按顺序进行训练,使机器人能够逐步扩展其现有的技能集。我们的核心思想是建立一个由神经网络表示的局部控制策略树,我们称之为中继神经网络。从尝试从一小组初始状态实现任务的根策略开始,每个后续策略通过将新状态驱动到现有的“良好”状态来扩展成功的初始状态集。我们的算法利用策略的价值函数来确定每个策略下的状态是否是“好”的。我们利用许多现有的策略搜索算法,学习价值函数的同时,政策,如那些使用演员-评论家表示或使用优势函数,以减少方差。我们证明了继电器网络可以解决欠驱动动态系统的复杂连续控制问题。

英文摘要:

While the recent advances in deep reinforcement learning have achieved impressive results in learning motor skills, many of the trained policies are only capable within a limited set of initial states. We propose a technique to break down a complex robotic task to simpler subtasks and train them sequentially such that the robot can expand its existing skill set gradually. Our key idea is to build a tree of local control policies represented by neural networks, which we refer as Relay Neural Networks. Starting from the root policy that attempts to achieve the task from a small set of initial states, each subsequent policy expands the set of successful initial states by driving the new states to existing "good" states. Our algorithm utilizes the value function of the policy to determine whether a state is "good" under each policy. We take advantage of many existing policy search algorithms that learn the value function simultaneously with the policy, such as those that use actor-critic representations or those that use the advantage function to reduce variance. We demonstrate that the relay networks can solve complex continuous control problems for underactuated dynamic systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑