arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.01529cs.ROcs.SYeess.SY
收藏

用于连续机器人控制的深度增量模型信息强化学习

Deep Incremental Model Informed Reinforcement Learning for Continuous Robotic Control

Cong Li

AI总结:

提出一步回看方法,联合学习深度增量模型和策略,降低模型学习难度,提高连续机器人控制的样本效率。

摘要:

基于模型的强化学习试图利用可用或学习到的模型来提高强化学习的数据效率。本文提出了一种一步回看方法,该方法联合学习深度增量模型和策略,以实现样本高效的连续机器人控制,其中利用控制理论知识来降低模型学习难度并促进高效训练。具体而言,我们使用一步向后数据来促进深度增量模型,这是一种机器人演化模型的替代结构化表示,能够准确预测机器人运动且具有较低的样本复杂度。这是因为所构建的深度增量模型将模型学习难度降为参数矩阵学习问题,这尤其有利于高维机器人应用。来自学习到的深度增量模型的想象数据被用来补充训练数据以增强样本效率。在基准连续机器人控制问题上进行了比较数值模拟,以验证我们提出的一步回看方法的有效性。

英文摘要:

Model-based reinforcement learning attempts to use an available or learned model to improve the data efficiency of reinforcement learning. This work proposes a one-step lookback approach that jointly learns the deep incremental model and the policy to realize the sample-efficient continuous robotic control, wherein the control-theoretical knowledge is utilized to decrease the model learning difficulty and facilitate efficient training. Specifically, we use one-step backward data to facilitate the deep incremental model, an alternative structured representation of the robotic evolution model, that accurately predicts the robotic movement but with low sample complexity. This is because the formulated deep incremental model degrades the model learning difficulty into a parametric matrix learning problem, which is especially favourable to high-dimensional robotic applications. The imagined data from the learned deep incremental model is used to supplement training data to enhance the sample efficiency. Comparative numerical simulations on benchmark continuous robotics control problems are conducted to validate the efficiency of our proposed one-step lookback approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑