In this paper, we study the problem of enabling a vision-based robotic manipulation system to generalize to novel tasks, a long-standing challenge in robot learning. We approach the challenge from an imitation learning perspective, aiming to study how scaling and broadening the data collected can facilitate such generalization. To that end, we develop an interactive and flexible imitation learning system that can learn from both demonstrations and interventions and can be conditioned on different forms of information that convey the task, including pre-trained embeddings of natural language or videos of humans performing the task. When scaling data collection on a real robot to more than 100 distinct tasks, we find that this system can perform 24 unseen manipulation tasks with an average success rate of 44%, without any robot demonstrations for those tasks.
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Hong Kong University of Science and Technology（香港科技大学）
Humans are capable of learning a new behavior by observing others perform the skill. Robots can also implement this by imitation learning. Furthermore, if with external guidance, humans will master the new behavior more efficiently. So how can robots implement this? To address the issue, we present Federated Imitation Learning (FIL) in the paper. Firstly, a knowledge fusion algorithm deployed on the cloud for fusing knowledge from local robots is presented. Then, effective transfer learning methods in FIL are introduced. With FIL, a robot is capable of utilizing knowledge from other robots to …
Ben-Gurion University of the Negev（内盖夫本-古里安大学）；Technical University Darmstadt（达姆施塔特工业大学）
The development of autonomous robotic systems that can learn from human demonstrations to imitate a desired behavior - rather than being manually programmed - has huge technological potential. One major challenge in imitation learning is the correspondence problem: how to establish corresponding states and actions between expert and learner, when the embodiments of the agents are different (morphology, dynamics, degrees of freedom, etc.). Many existing approaches in imitation learning circumvent the correspondence problem, for example, kinesthetic teaching or teleoperation, which are performed…
Imitation learning, and robot learning in general, emerged due to breakthroughs in machine learning, rather than breakthroughs in robotics. As such, evaluation metrics for robot learning are deeply rooted in those for machine learning, and focus primarily on data efficiency. We believe that a better metric for real-world robot learning is time efficiency, which better models the true cost to humans. This is a call to arms to the robot learning community to develop our own evaluation metrics, tailored towards the long-term goals of real-world robotics.
The transformation towards intelligence in various industries is creating more demand for intelligent and flexible products. In the field of robotics, learning-based methods are increasingly being applied, with the purpose of training robots to learn to deal with complex and changing external environments through data. In this context, reinforcement learning and imitation learning are becoming research hotspots with their respective characteristics. However, the two have their own limitations in some cases, such as the high cost of data acquisition for reinforcement learning. Moreover, it is d…
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Hong Kong University of Science and Technology（香港科技大学）；University of Macau（澳门大学）
Humans are capable of learning a new behavior by observing others to perform the skill. Similarly, robots can also implement this by imitation learning. Furthermore, if with external guidance, humans can master the new behavior more efficiently. So, how can robots achieve this? To address the issue, we present a novel framework named FIL. It provides a heterogeneous knowledge fusion mechanism for cloud robotic systems. Then, a knowledge fusion algorithm in FIL is proposed. It enables the cloud to fuse heterogeneous knowledge from local robots and generate guide models for robots with service r…
We address one-shot imitation learning, where the goal is to execute a previously unseen task based on a single demonstration. While there has been exciting progress in this direction, most of the approaches still require a few hundred tasks for meta-training, which limits the scalability of the approaches. Our main contribution is to formulate one-shot imitation learning as a symbolic planning problem along with the symbol grounding problem. This formulation disentangles the policy execution from the inter-task generalization and leads to better data efficiency. The key technical challenge is…
A general-purpose robot should be able to master a wide range of tasks and quickly learn a novel one by leveraging past experiences. One-shot imitation learning (OSIL) approaches this goal by training an agent with (pairs of) expert demonstrations, such that at test time, it can directly execute a new task from just one demonstration. However, so far this framework has been limited to training on many variations of one task, and testing on other unseen but similar variations of the same task. In this work, we push for a higher level of generalization ability by investigating a more ambitious m…
A Wearable Robotic Hand for Hand-over-Hand Imitation Learning
Tsinghua-Berkeley Shenzhen Institute, Tsinghua University（清华大学清华-伯克利深圳学院）；Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）；Shanghai AI Lab（上海人工智能实验室）；Shanghai Qi Zhi Institute（上海期智研究院）
RoboManipBaselines: A Unified Framework for Imitation Learning in Robotic Manipulation across Real and Simulation Environments
Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology (AIST)（日本国家先进工业科学技术研究院人工智能研究中心）；CNRS-AIST JRL (Joint Robotics Laboratory), IRL（法国国家科学研究中心与日本AIST联合机器人实验室）；Institute for AI and Robotics, Future Robotics Organization, Waseda University（早稻田大学未来机器人组织人工智能与机器人研究所）；AI Robot Association (AIRoA)（人工智能机器人协会）
Northeastern University（东北大学）；Faculty of Robot Science and Engineering, Northeastern University（东北大学机器人科学与工程学院）；College of Information Science and Engineering, Northeastern University（东北大学信息科学与工程学院）；Foshan Graduate School, Northeastern University（东北大学佛山研究生院）
University of Tokyo（东京大学）；Technische Universität Darmstadt（达姆施塔特工业大学）；University of Lincoln（林肯大学）；Carnegie Mellon University（卡内基梅隆大学）；University of California, Berkeley（加州大学伯克利分校）
As robots and other intelligent agents move from simple environments and problems to more complex, unstructured settings, manually programming their behavior has become increasingly challenging and expensive. Often, it is easier for a teacher to demonstrate a desired behavior rather than attempt to manually engineer it. This process of learning from demonstrations, and the study of algorithms to do so, is called imitation learning. This work provides an introduction to imitation learning. It covers the underlying assumptions, approaches, and how they relate; the rich set of algorithms develope…
University of Maryland（马里兰大学）；Syracuse University（雪城大学）
Imitation learning allows social robots to learn new skills from human teachers without substantial manual programming, but it is difficult for robotic imitation learning systems to generalize demonstrated skills as well as human learners do. Contemporary neurocomputational approaches to imitation learning achieve limited generalization at the cost of data-intensive training, and often produce opaque models that are difficult to understand and debug. In this study, we explore the viability of developing purely-neural controllers for social robots that learn to imitate by reasoning about the un…
University of Bristol（布里斯托大学）；University of the West of England（西英格兰大学）；Bristol Robotic Laboratory（布里斯托机器人实验室）
Traditional deep learning-based visual imitation learning techniques require a large amount of demonstration data for model training, and the pre-trained models are difficult to adapt to new scenarios. To address these limitations, we propose a unified framework using a novel progressive learning approach comprised of three phases: i) a coarse learning phase for concept representation, ii) a fine learning phase for action generation, and iii) an imaginary learning phase for domain adaptation. Overall, this approach leads to a one-shot domain-adaptive imitation learning framework. We use roboti…
DAISS: Phase-Aware Imitation Learning for Dual-Arm Robotic Ultrasound-Guided Interventions
Chair for Computer Aided Medical Procedures(CAMP), Technical University of Munich(TUM)（计算机辅助医疗程序主席职位（CAMP），慕尼黑技术大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；The University of Hong Kong(HKU)（香港大学）
Osaka University（大阪大学）；National Institute of Advanced Industrial Science and Technology（国立先进工业科学技术研究院）
Robotic assembly tasks involve complex and low-clearance insertion trajectories with varying contact forces at different stages. While the nominal motion trajectory can be easily obtained from human demonstrations through kinesthetic teaching, teleoperation, simulation, among other methods, the force profile is harder to obtain especially when a real robot is unavailable. It is difficult to obtain a realistic force profile in simulation even with physics engines. Such simulated force profiles tend to be unsuitable for the actual robotic assembly due to the reality gap and uncertainty in the as…
Robotic Manipulation via Imitation Learning: Taxonomy, Evolution, Benchmark, and Challenges
Ecole Centrale de Lyon, CNRS, Universite Claude Bernard Lyon 1, INSA Lyon, Université Lumière Lyon 2, LIRIS, UMR5205（Lyon中央理工学院、CNRS、Claude Bernard Lyon 1大学、INSA Lyon、Lyon大学、LIRIS、UMR5205）；Beihang University（北航大学）；Dalian University of Technology（大连理工大学）；Institut Universitaire de France (IUF)（法国大学研究院（IUF））
Mastering Contact-rich Tasks by Combining Soft and Rigid Robotics with Imitation Learning
Cognitive Robotics, Delft University of Technology（德鲁特理工大学认知机器人学）；Bioinspired Soft Robotics Lab, Istituto Italiano di Tecnologia（意大利技术研究院仿生软机器人实验室）；Technology Innovation Institute (TII)（技术创新研究所）；Institute of Robotics and Mechatronics, German Aerospace Center (DLR)（德国航空航天中心机器人与机电研究所）
In recent years, we have seen an emergence of data-driven approaches in robotics. However, most existing efforts and datasets are either in simulation or focus on a single task in isolation such as grasping, pushing or poking. In order to make progress and capture the space of manipulation, we would need to collect a large-scale dataset of diverse tasks such as pouring, opening bottles, stacking objects etc. But how does one collect such a dataset? In this paper, we present the largest available robotic-demonstration dataset (MIME) that contains 8260 human-robot demonstrations over 20 differen…
University of California, Berkeley（加州大学伯克利分校）；OpenAI
In order for a robot to be a generalist that can perform a wide range of jobs, it must be able to acquire a wide variety of skills quickly and efficiently in complex unstructured environments. High-capacity models such as deep neural networks can enable a robot to represent complex skills, but learning each skill from scratch then becomes infeasible. In this work, we present a meta-imitation learning method that enables a robot to learn how to learn more efficiently, allowing it to acquire new skills from just a single demonstration. Unlike prior methods for one-shot imitation, our method can …
Gaze-Guided Task Decomposition for Imitation Learning in Robotic Manipulation
The University of Tokyo（东京大学）；Graduate School of Information Science and Technology（信息科学与技术研究生院）；Laboratory for Intelligent Systems and Informatics（智能系统与信息学实验室）
Nara Institute of Science and Technology（奈良先端科学技术大学院大学）
Generative Adversarial Imitation Learning (GAIL) can learn policies without explicitly defining the reward function from demonstrations. GAIL has the potential to learn policies with high-dimensional observations as input, e.g., images. By applying GAIL to a real robot, perhaps robot policies can be obtained for daily activities like washing, folding clothes, cooking, and cleaning. However, human demonstration data are often imperfect due to mistakes, which degrade the performance of the resulting policies. We address this issue by focusing on the following features: 1) many robotic tasks are …
Contrastive Imitation Learning for Language-guided Multi-Task Robotic Manipulation
AI Thrust, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）AI Thrust）；The Hong Kong University of Science and Technology（香港科技大学）
Imperial College London（伦敦帝国理工学院）；Tencent Robotics X（腾讯 Robotics X）；Bielefeld University（比勒费尔德大学）
To accurately pour drinks into various containers is an essential skill for service robots. However, drink pouring is a dynamic process and difficult to model. Traditional deep imitation learning techniques for implementing autonomous robotic pouring have an inherent black-box effect and require a large amount of demonstration data for model training. To address these issues, an Explainable Hierarchical Imitation Learning (EHIL) method is proposed in this paper such that a robot can learn high-level general knowledge and execute low-level actions across multiple drink pouring scenarios. Moreov…
Improving Robotic Imitation Learning via Trajectory Standardization
State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所多模态人工智能系统国家重点实验室）；D-Robotics
University of Southern California（南加州大学）；Meta AI；KAIST（韩国科学技术院）；Georgia Tech（佐治亚理工学院）
We propose an approach for semantic imitation, which uses demonstrations from a source domain, e.g. human videos, to accelerate reinforcement learning (RL) in a different target domain, e.g. a robotic manipulator in a simulated kitchen. Instead of imitating low-level actions like joint velocities, our approach imitates the sequence of demonstrated semantic skills like "opening the microwave" or "turning on the stove". This allows us to transfer demonstrations across environments (e.g. real-world to simulated kitchen) and agent embodiments (e.g. bimanual human demonstration to robotic arm). We …
