arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1907.00526cs.LGcs.AIstat.ML
收藏

FiDi-RL:用差分策略搜索实现深度强化学习,以实现连续控制的有效学习

FiDi-RL: Incorporating Deep Reinforcement Learning with Finite-Difference Policy Search for Efficient Learning of Continuous Control

Longxiang Shi, Shijian Li, Longbing Cao, Long Yang, Gang Zheng, Gang Pan

摘要:

近年来,强化学习在处理复杂控制问题方面取得了很大进展,但强化学习在连续控制任务中仍面临挑战。传统的方法计算最优目标的导数需要耗费大量的计算资源,且效率低、不稳定、鲁棒性差。或者,基于导数的方法将优化过程视为黑盒,并在学习连续控制任务时显示出鲁棒性和稳定性,但在学习中没有数据效率。如何将这两种方法结合起来,以达到两者的最佳效果,已经引起了人们的关注。然而,大多数现有的组合工程采用复杂的神经网络(NN)作为控制策略。深度神经网络的双刃剑可以产生更好的性能,但也使得参数调整和计算变得困难。为此,本文提出了一种新的方法FiDi-RL,该方法将深度强化学习与基于策略差异的策略搜索相结合,将深度确定性策略梯度(DDPG)与扩充随机搜索(ARS)相结合,旨在提高ARS的数据效率。实验结果表明,FiDi-RL可以提高ARS的性能和稳定性,并提供与现有的一些深度强化学习方法相竞争的结果

英文摘要:

In recent years significant progress has been made in dealing with challenging problems using reinforcement learning.Despite its great success, reinforcement learning still faces challenge in continuous control tasks. Conventional methods always compute the derivatives of the optimal goal with a costly computation resources, and are inefficient, unstable and lack of robust-ness when dealing with such tasks. Alternatively, derivative-based methods treat the optimization process as a blackbox and show robustness and stability in learning continuous control tasks, but not data efficient in learning. The combination of both methods so as to get the best of the both has raised attention. However, most of the existing combination works adopt complex neural networks (NNs) as the policy for control. The double-edged sword of deep NNs can yield better performance, but also makes it difficult for parameter tuning and computation. To this end, in this paper we presents a novel method called FiDi-RL, which incorporates deep RL with Finite-Difference (FiDi) policy search.FiDi-RL combines Deep Deterministic Policy Gradients (DDPG)with Augment Random Search (ARS) and aims at improving the data efficiency of ARS. The empirical results show that FiDi-RL can improves the performance and stability of ARS, and provide competitive results against some existing deep reinforcement learning methods

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑