具有不同传感器的模拟机器人之间的迁移学习
Transfer Learning Across Simulated Robots With Different Sensors
摘要:
对于机器人来说,要学习一个好的策略,它通常需要昂贵的设备(如复杂的传感器)和一个有利于学习的准备好的训练环境。然而,由于经济原因,很少有可能完美地装备机器人,也不能保证理想的学习条件,当部署在现实生活环境中时。一个解决方案是在实验室环境中准备机器人,当所有必要的材料都可以用来学习一个好的策略时。经过实验室训练后,机器人应该能够在没有昂贵设备的情况下使用,但仍然可以保证在现场表现良好。实验室(源)和真实世界环境(目标)之间的转换与迁移学习有关,其中源任务和目标任务之间的状态空间不同。我们使用Bootstrapped Dual Policy Iteration,一种无模型的Actor-Critic强化学习算法和Policy Shaping来解决具有连续状态和离散动作的模拟任务。具体来说,我们训练一个BDPI代理,体现了一个虚拟机器人执行任务的V-Rep模拟器,通过几个接近传感器感测其环境。然后,第二个智能体使用所产生的策略在相同的环境中学习相同的任务,但使用相机图像作为输入。我们的目标是获得一个策略,能够执行任务,仅仅依靠相机图像。
英文摘要:
For a robot to learn a good policy, it often requires expensive equipment (such as sophisticated sensors) and a prepared training environment conducive to learning. However, it is seldom possible to perfectly equip robots for economic reasons, nor to guarantee ideal learning conditions, when deployed in real-life environments. A solution would be to prepare the robot in the lab environment, when all necessary material is available to learn a good policy. After training in the lab, the robot should be able to get by without the expensive equipment that used to be available to it, and yet still be guaranteed to perform well on the field. The transition between the lab (source) and the real-world environment (target) is related to transfer learning, where the state-space between the source and target tasks differ. We tackle a simulated task with continuous states and discrete actions presenting this challenge, using Bootstrapped Dual Policy Iteration, a model-free actor-critic reinforcement learning algorithm, and Policy Shaping. Specifically, we train a BDPI agent, embodied by a virtual robot performing a task in the V-Rep simulator, sensing its environment through several proximity sensors. The resulting policy is then used by a second agent learning the same task in the same environment, but with camera images as input. The goal is to obtain a policy able to perform the task relying on merely camera images.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。