自动调谐的仿真到现实迁移
Auto-Tuned Sim-to-Real Transfer
摘要:
在模拟中训练的政策在转移到现实世界时往往会失败,这是由于“现实差距”,模拟器无法准确捕捉现实世界的动态和视觉特性。当前解决这个问题的方法,例如域随机化,需要先验知识和工程来确定将系统参数随机化多少,以便学习对仿真到现实迁移鲁棒同时也不太保守的策略。我们提出了一种自动调整模拟器系统参数,以匹配真实世界的方法,只使用原始RGB图像的真实世界,而不需要定义奖励或估计状态。我们的主要观点是将参数的自动调整重新定义为一个搜索问题,其中我们迭代地移动仿真系统参数以接近真实世界的系统参数。我们提出了一个搜索参数模型(SPM),给定一个序列的观察和行动和一组系统参数,预测给定的参数是高于还是低于用于生成观察的真实参数。我们评估我们的方法在多个机器人控制任务,在模拟到模拟和仿真到现实迁移,表现出显着的改善幼稚域随机化。项目视频和代码https://yuqingd.github.io/autotuned-sim2real/
英文摘要:
Policies trained in simulation often fail when transferred to the real world due to the `reality gap' where the simulator is unable to accurately capture the dynamics and visual properties of the real world. Current approaches to tackle this problem, such as domain randomization, require prior knowledge and engineering to determine how much to randomize system parameters in order to learn a policy that is robust to sim-to-real transfer while also not being too conservative. We propose a method for automatically tuning simulator system parameters to match the real world using only raw RGB images of the real world without the need to define rewards or estimate state. Our key insight is to reframe the auto-tuning of parameters as a search problem where we iteratively shift the simulation system parameters to approach the real-world system parameters. We propose a Search Param Model (SPM) that, given a sequence of observations and actions and a set of system parameters, predicts whether the given parameters are higher or lower than the true parameters used to generate the observations. We evaluate our method on multiple robotic control tasks in both sim-to-sim and sim-to-real transfer, demonstrating significant improvement over naive domain randomization. Project videos and code at https://yuqingd.github.io/autotuned-sim2real/
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。