arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1610.03518cs.ROcs.AIcs.LGcs.SYeess.SY
收藏

通过学习深度逆动力学模型实现仿真到现实世界的迁移

Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model

OpenAI(开放人工智能研究院)

Paul Christiano, Zain Shah, Igor Mordatch, Jonas Schneider, Trevor Blackwell, Joshua Tobin, Pieter Abbeel, Wojciech Zaremba

AI总结:

本文提出通过学习深度逆动力学模型,在仿真与现实世界之间实现控制策略的迁移,解决仿真与现实差异导致的性能下降问题。

摘要:

在仿真中开发控制策略通常比直接在现实世界中运行实验更实用和安全。这适用于通过规划和优化获得的策略,甚至更适用于通过强化学习获得的策略,后者通常非常数据密集。然而,仿真中成功的策略在部署到现实机器人时往往无法工作。然而,策略在仿真中执行的整体思路在现实世界中通常仍然有效。本文研究了此类场景,其中仿真中遍历的状态序列在现实世界中仍然合理,即使控制细节不同,例如摩擦、接触、质量和几何属性的差异。在执行过程中,我们的方法在每个时间步计算仿真基于的控制策略会做什么,但不执行这些控制在现实机器人上,而是计算仿真期望的下一个状态,并依赖于学习的深度逆动力学模型来决定最合适的现实世界动作以达到这些状态。深度模型只有在训练数据足够的情况下才有效,我们还提出了一种数据收集方法来(逐步)学习深度逆动力学模型。我们的实验表明,我们的方法在处理仿真到现实世界模型差异的各种基线方法中表现良好,包括输出误差控制和高斯动态适应。

英文摘要:

Developing control policies in simulation is often more practical and safer than directly running experiments in the real world. This applies to policies obtained from planning and optimization, and even more so to policies obtained from reinforcement learning, which is often very data demanding. However, a policy that succeeds in simulation often doesn't work when deployed on a real robot. Nevertheless, often the overall gist of what the policy does in simulation remains valid in the real world. In this paper we investigate such settings, where the sequence of states traversed in simulation remains reasonable for the real world, even if the details of the controls are not, as could be the case when the key differences lie in detailed friction, contact, mass and geometry properties. During execution, at each time step our approach computes what the simulation-based control policy would do, but then, rather than executing these controls on the real robot, our approach computes what the simulation expects the resulting next state(s) will be, and then relies on a learned deep inverse dynamics model to decide which real-world action is most suitable to achieve those next states. Deep models are only as good as their training data, and we also propose an approach for data collection to (incrementally) learn the deep inverse dynamics model. Our experiments shows our approach compares favorably with various baselines that have been developed for dealing with simulation to real world model discrepancy, including output error control and Gaussian dynamics adaptation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑