This paper focuses on the scalable robot learning for manipulation in the dexterous robot arm-hand systems, where the remote human-robot interactions via augmented reality (AR) are established to collect the expert demonstration data for improving efficiency. In such a system, we present a novel method to address the general manipulation task problem. Specifically, the proposed method consists of two phases: i) In the first phase for pretraining, the policy is created in a behavior cloning (BC) manner, through leveraging the learning data from our AR-based remote human-robot interaction system; ii) In the second phase, a contrastive learning empowered reinforcement learning (RL) method is developed to obtain more efficient and robust policy than the BC, and thus a projection head is designed to accelerate the learning progress. An event-driven augmented reward is adopted for enhancing the safety. To validate the proposed method, both the physics simulations via PyBullet and real-world experiments are carried out. The results demonstrate that compared to the baselines, our method not only significantly speeds up the training process, but also achieves much better performance in terms of the success rate for fulfilling the manipulation tasks. By conducting the ablation study, it is confirmed that the proposed RL with contrastive learning overcomes policy collapse. Supplementary demonstrations are available at https://cyberyyc.github.io/.
TACROSS: An Efficient and Low-Cost Scalable Human Touch System Across Heterogeneous Tactile Sensors for Dexterous Robot Learning
The University of Hong Kong（香港大学）；INFIFORCE；The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；Wuhan Textile University（武汉纺织大学）；Ocean University of China（中国海洋大学）；Wuhan University（武汉大学）
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Aalto University（阿alto大学）；University of Southern California（南加州大学）；Imperial College London（伦敦帝国理工学院）；University of Oulu（奥卢大学）；University of Alberta（阿尔伯塔大学）；Alberta Machine Intelligence Institute（阿尔伯塔人工智能研究所）
DexEXO: A Wearability-First Dexterous Exoskeleton for Operator-Agnostic Demonstration and Learning
Department of Computer Science（计算机科学系）；Department of Electrical and Computer Engineering（电气与计算机工程系）；Department of Mechanical and Aerospace Engineering（机械与航空航天工程系）
Deep learning and reinforcement learning methods have recently been used to solve a variety of problems in continuous control domains. An obvious application of these techniques is dexterous manipulation tasks in robotics which are difficult to solve using traditional control theory or hand-engineered approaches. One example of such a task is to grasp an object and precisely stack it on another. Solving this difficult and practically relevant problem in the real world is an important long-term goal for the field of robotics. Here we take a step towards this goal by examining the problem in sim…
UC Berkeley（加州大学伯克利分校）；University of Washington（华盛顿大学）
Reinforcement Learning (RL) algorithms can in principle acquire complex robotic skills by learning from large amounts of data in the real world, collected via trial and error. However, most RL algorithms use a carefully engineered setup in order to collect data, requiring human supervision and intervention to provide episodic resets. This is particularly evident in challenging robotics problems, such as dexterous manipulation. To make data collection scalable, such applications require reset-free algorithms that are able to learn autonomously, without explicit instrumentation or human interven…
Real Robot Challenge 2022: Learning Dexterous Manipulation from Offline Data in the Real World
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Helmholtz and TU Munich（亥姆霍兹与慕尼黑工业大学）；Harvard University（哈佛大学）；DeepMind（深度思维）；University of Toronto（多伦多大学）；University College Dublin（都柏林大学学院）；Pohang University of Science and Technology（浦项科技大学）；Electronics and Telecommunications Research Institute（电子与电信研究所）；ETH Zurich（苏黎世联邦理工学院）
本文介绍了 Real Robot Challenge 2022，该竞赛通过提供真实机器人离线数据集和远程评估平台，促进了强化学习与机器人社区的交流，并比较了获胜团队与最先进离线 RL 算法在灵巧操作任务上的表现。
Remote sensing can provide crucial information for planetary rovers. However, they must validate these orbital observations with in situ measurements. Typically, this involves validating hyperspectral data using a spectrometer on-board the field robot. In order to achieve this, the robot must visit sampling locations that jointly improve a model of the environment while satisfying sampling constraints. However, current planners follow sub-optimal greedy strategies that are not scalable to larger regions. We demonstrate how the problem can be effectively defined in an MDP framework and propose …
University of California, Berkeley（加州大学伯克利分校）；University of Oxford（牛津大学）；University of Washington（华盛顿大学）
The success of reinforcement learning for real world robotics has been, in many cases limited to instrumented laboratory scenarios, often requiring arduous human effort and oversight to enable continuous learning. In this work, we discuss the elements that are needed for a robotic learning system that can continually and autonomously improve with data collected in the real world. We propose a particular instantiation of such a system, using dexterous manipulation as our case study. Subsequently, we investigate a number of challenges that come up when learning without instrumentation. In such s…
