复杂动力学机器人任务中基于鲁棒控制器的仿真到现实迁移学习
Sim-to-Real Transfer Learning using Robustified Controllers in Robotic Tasks involving Complex Dynamics
摘要:
使用深度强化学习学习机器人任务或控制器已在模拟中被证明是有效的。模拟学习有几个优点。例如,可以完全控制模拟环境,包括在执行计算时停止运动。机器人参与的另一个优点是,通过将学习到的任务转移到真正的机器人上,可以减少机器人学习任务而不是生产的时间。迁移学习需要对真实机器人进行一定程度的微调。对于涉及复杂(非线性)动态的任务,微调本身可能花费大量时间。为了减少微调量,我们建议在仿真中学习鲁棒控制器。鲁棒控制器是通过利用连续训练集改变仿真参数(外观和动态)的能力来学习的。这种方法的另一个好处是,它简化了模拟器的物理参数的精确确定,这是一个不平凡的任务。我们展示了我们提出的方法在一个真正的设置,其中机器人的目的是解决迷宫游戏,这涉及到复杂的动态,由于静摩擦和潜在的大加速度。我们表明,在迁移学习的鲁棒控制器的微调量大幅减少相比,非鲁棒控制器。
英文摘要:
Learning robot tasks or controllers using deep reinforcement learning has been proven effective in simulations. Learning in simulation has several advantages. For example, one can fully control the simulated environment, including halting motions while performing computations. Another advantage when robots are involved, is that the amount of time a robot is occupied learning a task---rather than being productive---can be reduced by transferring the learned task to the real robot. Transfer learning requires some amount of fine-tuning on the real robot. For tasks which involve complex (non-linear) dynamics, the fine-tuning itself may take a substantial amount of time. In order to reduce the amount of fine-tuning we propose to learn robustified controllers in simulation. Robustified controllers are learned by exploiting the ability to change simulation parameters (both appearance and dynamics) for successive training episodes. An additional benefit for this approach is that it alleviates the precise determination of physics parameters for the simulator, which is a non-trivial task. We demonstrate our proposed approach on a real setup in which a robot aims to solve a maze game, which involves complex dynamics due to static friction and potentially large accelerations. We show that the amount of fine-tuning in transfer learning for a robustified controller is substantially reduced compared to a non-robustified controller.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。