arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2101.06005cs.RO
收藏

SimGAN:基于对抗强化学习的领域自适应混合模拟器识别

SimGAN: Hybrid Simulator Identification for Domain Adaptation via Adversarial Reinforcement Learning

Google(谷歌);Stanford University(斯坦福大学);Everyday Robots(日常机器人公司);X The Moonshot Factory(X登月工厂);University of California, Berkeley(加州大学伯克利分校)

Yifeng Jiang, Tingnan Zhang, Daniel Ho, Yunfei Bai, C. Karen Liu, Sergey Levine, Jie Tan

摘要:

随着基于学习的方法朝着自动化机器人控制器设计的方向发展,将学习到的策略转移到具有不同动态的新域(例如,仿真到现实迁移)仍然需要人工努力。本文介绍了SimGAN,一个框架,以解决领域自适应通过识别一个混合物理模拟器,以匹配模拟轨迹的目标域,使用学习判别损失,以解决与手动损失设计的局限性。我们的混合模拟器结合了神经网络和传统的物理模拟,以平衡表现力和泛化能力,并强调了需要在系统ID中精心选择的参数集。一旦通过对抗性强化学习识别混合模拟器,它可以用于改进目标域的策略,而不需要交错数据汇集和策略改进。我们表明,我们的方法优于多个强基线的领域自适应六个机器人运动任务。

英文摘要:

As learning-based approaches progress towards automating robot controllers design, transferring learned policies to new domains with different dynamics (e.g. sim-to-real transfer) still demands manual effort. This paper introduces SimGAN, a framework to tackle domain adaptation by identifying a hybrid physics simulator to match the simulated trajectories to the ones from the target domain, using a learned discriminative loss to address the limitations associated with manual loss design. Our hybrid simulator combines neural networks and traditional physics simulation to balance expressiveness and generalizability, and alleviates the need for a carefully selected parameter set in System ID. Once the hybrid simulator is identified via adversarial reinforcement learning, it can be used to refine policies for the target domain, without the need to interleave data collection and policy refinement. We show that our approach outperforms multiple strong baselines on six robotic locomotion tasks for domain adaptation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑