Learning robot control policies from physics simulations is of great interest to the robotics community as it may render the learning process faster, cheaper, and safer by alleviating the need for expensive real-world experiments. However, the direct transfer of learned behavior from simulation to reality is a major challenge. Optimizing a policy on a slightly faulty simulator can easily lead to the maximization of the `Simulation Optimization Bias` (SOB). In this case, the optimizer exploits modeling errors of the simulator such that the resulting behavior can potentially damage the robot. We tackle this challenge by applying domain randomization, i.e., randomizing the parameters of the physics simulations during learning. We propose an algorithm called Simulation-based Policy Optimization with Transferability Assessment (SPOTA) which uses an estimator of the SOB to formulate a stopping criterion for training. The introduced estimator quantifies the over-fitting to the set of domains experienced while training. Our experimental results on two different second order nonlinear systems show that the new simulation-based policy search algorithm is able to learn a control policy exclusively from a randomized simulator, which can be applied directly to real systems without any additional training.
Crossing the Reality Gap: a Short Introduction to the Transferability Approach
Université Pierre et Marie Curie-Paris 6(皮埃尔和玛丽·居里大学-巴黎第六大学);CNRS UMR 7222(法国国家科学研究中心UMR 7222实验室)
在机器人技术中,无梯度优化算法(例如进化算法)通常只用于仿真,因为它们需要评估许多候选解。然而,在仿真中获得的解决方案往往不能很好地工作在真实设备上。可移植性方法旨在通过\emph{making the optimization algorithm aware of the limits of the simulation}跨越仿真和现实之间的这一差距。在本文中,我们首先描述的可转移性功能,解决方案的描述符映射到一个分数代表如何以及模拟器匹配的现实。然后,我们表明,这个功能可以使用回归算法和一些实验与真实的设备。我们的研究结果是支持一个简单的四足机器人,其控制参数进行了优化的现实差距的广泛研究。特别是,我们在现实和模拟中映射了整个搜索空间,以了解适应度景观之间的差异。
MIHRaGe: A Mixed-Reality Interface for Human-Robot Interaction via Gaze-Oriented Control
Engineering School of São Carlos University of São Paulo(圣保罗大学圣卡洛斯工程学院);Massachussets Institute of Technology(麻省理工学院);Hospital Israelita Albert Einstein(以色列医院阿尔伯特·爱因斯坦医院)
Immersive Explainability: Visualizing Robot Navigation Decisions through XAI Semantic Scene Projections in Virtual Reality
University of Bonn(波恩大学);Lamarr Institute for Machine Learning and Artificial Intelligence(拉马尔人工智能与机器学习研究所);Center for Robotics(机器人中心);University of Mainz(美因茨大学);Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS(弗劳恩霍夫智能分析与信息系统研究所)