Imitation learning is an effective tool for robotic learning tasks where specifying a reinforcement learning (RL) reward is not feasible or where the exploration problem is particularly difficult. Imitation, typically behavior cloning or inverse RL, derive a policy from a collection of first-person action-state trajectories. This is contrary to how humans and other animals imitate: we observe a behavior, even from other species, understand its perceived effect on the state of the environment, and figure out what actions our body can perform to reach a similar outcome. In this work, we explore the possibility of third-person visual imitation of manipulation trajectories, only from vision and without access to actions, demonstrated by embodiments different to the ones of our imitating agent. Specifically, we investigate what would be an appropriate representation method with which an RL agent can visually track trajectories of complex manipulation behavior -- non-planar with multiple-object interactions -- demonstrated by experts with different embodiments. We present a way to train manipulator-independent representations (MIR) that primarily focus on the change in the environment and have all the characteristics that make them suitable for cross-embodiment visual imitation with RL: cross-domain alignment, temporal smoothness, and being actionable. We show that with our proposed method our agents are able to imitate, with complex robot control, trajectories from a variety of embodiments and with significant visual and dynamics differences, e.g. simulation-to-reality gap.
Yaskawa India Pvt. Ltd.（安川印度私人有限公司）；Robert Bosch Center for Cyber-Physical Systems, Indian Institute of Science（印度科学学院罗伯特·博世信息物理系统中心）
Industrial robot manipulators are not able to match the precision and speed with which humans are able to execute contact rich tasks even to this day. Therefore, as a means overcome this gap, we demonstrate generative methods for imitating a peg-in-hole insertion task in a 6-DOF robot manipulator. In particular, generative adversarial imitation learning (GAIL) is used to successfully achieve this task with a 10 um, and a 6 um peg-hole clearance on the Yaskawa GP8 industrial robot. Experimental results show that the policy successfully learns within 20 episodes from a handful of human expert de…
Imitation Learning-based Visual Servoing for Tracking Moving Objects
Università della Svizzera italiana (USI)（瑞士意大利语区大学（USI））；University of Trento（特伦托大学）；Dalle Molle Institute for Artificial Intelligence (IDSIA)（达勒莫勒人工智能研究所（IDSIA））
University of Maryland at College Park（马里兰大学帕克分校）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；City University of Hong Kong（香港城市大学）；the University of Hong Kong（香港大学）
We present a novel approach for robust manipulation of high-DOF deformable objects such as cloth. Our approach uses a random forest-based controller that maps the observed visual features of the cloth to an optimal control action of the manipulator. The topological structure of this random forest-based controller is determined automatically based on the training data consisting visual features and optimal control actions. This enables us to integrate the overall process of training data classification and controller optimization into an imitation learning (IL) approach. Our approach enables le…
Design and Experimental Validation of Sensorless 4-Channel Bilateral Teleoperation for Low-Cost Manipulators
Degree Programs in Intelligent and Mechanical Interaction Systems, University of Tsukuba（智能与机械交互系统专业，东京大学）；Faculty of Engineering, Information and Systems, University of Tsukuba（工程、信息与系统学部，东京大学）；Department of Electrical Engineering, Electronics, and Applied Physics, Saitama University（电子工程、电子学与应用物理系，埼玉大学）
Tsinghua University（清华大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Pennsylvania（宾夕法尼亚大学）；Shanghai Qi Zhi Institute（上海期智研究院）
Imitation learning trains control policies by mimicking pre-recorded expert demonstrations. In partially observable settings, imitation policies must rely on observation histories, but many seemingly paradoxical results show better performance for policies that only access the most recent observation. Recent solutions ranging from causal graph learning to deep information bottlenecks have shown promising results, but failed to scale to realistic settings such as visual imitation. We propose a solution that outperforms these prior approaches by upweighting demonstration keyframes corresponding …
GrOMP: Grasped Object Manifold Projection for Multimodal Imitation Learning of Manipulation
Mechanical Engineering Department at the University of Michigan（密歇根大学机械工程系）；Robotics & Automation Research Department at the Ford Motor Company（福特汽车公司机器人与自动化研究部）；Robotics Department at the University of Michigan（密歇根大学机器人系）
NVIDIA（英伟达）；Georgia Institute of Technology（佐治亚理工学院）；University of Washington（华盛顿大学）
Imitation learning (IL) is a frequently used approach for data-efficient policy learning. Many IL methods, such as Dataset Aggregation (DAgger), combat challenges like distributional shift by interacting with oracular experts. Unfortunately, assuming access to oracular experts is often unrealistic in practice; data used in IL frequently comes from offline processes such as lead-through or teleoperation. In this paper, we present a novel imitation learning technique called Collocation for Demonstration Encoding (CoDE) that operates on only a fixed set of trajectory demonstrations. We circumvent…
OSVI-WM: One-Shot Visual Imitation for Unseen Tasks using World-Model-Guided Trajectory Generation
New York University Tandon School of Engineering（纽约大学Tandon工程学院）；New York University Courant Institute of Mathematical Sciences（纽约大学Courant数学科学研究所）；Meta-FAIR
University of Surrey（萨里大学）；Karlshule Institute of Technology（卡尔斯鲁厄理工学院）
This work adds on to the on-going efforts to provide more autonomy to space robots. Here the concept of programming by demonstration or imitation learning is used for trajectory planning of manipulators mounted on small spacecraft. For greater autonomy in future space missions and minimal human intervention through ground control, a robot arm having 7-Degrees of Freedom (DoF) is envisaged for carrying out multiple tasks like debris removal, on-orbit servicing and assembly. Since actual hardware implementation of microgravity environment is extremely expensive, the demonstration data for trajec…
FZI Research Center for Information Technology（FZI信息科技研究中心）；IAR Institute for Anthropomatics and Robotics, HIS Humanoids and Intelligence Systems Lab, KIT Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院（KIT）IAR人形机器人与机器人学研究所、HIS人形与智能系统实验室）
Robotic automation is a key driver for the advancement of technology. The skills of human workers, however, are difficult to program and seem currently unmatched by technical systems. In this work we present a data-driven approach to extract and learn robot-independent contact skills from human demonstrations in simulation environments, using a Long Short Term Memory (LSTM) network. Our model learns to generate error-correcting sequences of forces and torques in task space from object-relative motion, which industrial robots carry out through a Cartesian force control scheme on the real setup.…
We consider the problem of visual imitation learning without human supervision (e.g. kinesthetic teaching or teleoperation), nor access to an interactive reinforcement learning (RL) training environment. We present a geometric perspective to derive solutions to this problem. Specifically, we propose VGS-IL (Visual Geometric Skill Imitation Learning), an end-to-end geometry-parameterized task concept inference method, to infer globally consistent geometric feature association rules from human demonstration video frames. We show that, instead of learning actions from image pixels, learning a geo…
RTFF: Random-to-Target Fabric Flattening Policy using Dual-Arm Manipulator
Department of Electrical and Electronic Engineering, Faculty of Engineering, The University of Hong Kong（香港大学电子与电气工程系）；Dyson School of Design Engineering, Imperial College London（帝国理工学院设计工程学院）；Unprecedented-scale Data Analytics Center, Tohoku University（东北大学大规模数据分析中心）；Graduate School of Information Sciences, Tohoku University（东北大学信息科学研究生院）；Department of Mechanical Engineering, City University of Hong Kong（香港城市大学机械工程系）
ViTac-Tracing: Visual-Tactile Imitation Learning of Deformable Object Tracing
Robot Perception Laboratory, Department of Engineering, King’s College London（机器人感知实验室，工程系，伦敦国王学院）；Personal Robotics Laboratory, Department of Electrical and Electronic Engineering, Imperial College London（个人机器人实验室，电子与电气工程系，伦敦帝国学院）
While visual imitation learning offers one of the most effective ways of learning from visual demonstrations, generalizing from them requires either hundreds of diverse demonstrations, task specific priors, or large, hard-to-train parametric models. One reason such complexities arise is because standard visual imitation frameworks try to solve two coupled problems at once: learning a succinct but good representation from the diverse visual data, while simultaneously learning to associate the demonstrated actions with such representations. Such joint learning causes an interdependence between t…
Mastering Contact-rich Tasks by Combining Soft and Rigid Robotics with Imitation Learning
Cognitive Robotics, Delft University of Technology（德鲁特理工大学认知机器人学）；Bioinspired Soft Robotics Lab, Istituto Italiano di Tecnologia（意大利技术研究院仿生软机器人实验室）；Technology Innovation Institute (TII)（技术创新研究所）；Institute of Robotics and Mechatronics, German Aerospace Center (DLR)（德国航空航天中心机器人与机电研究所）
We propose to perform imitation learning for dexterous manipulation with multi-finger robot hand from human demonstrations, and transfer the policy to the real robot hand. We introduce a novel single-camera teleoperation system to collect the 3D demonstrations efficiently with only an iPad and a computer. One key contribution of our system is that we construct a customized robot hand for each user in the physical simulator, which is a manipulator resembling the same kinematics structure and shape of the operator's hand. This provides an intuitive interface and avoid unstable human-robot hand r…
Learning to Turn: Diffusion Imitation for Robust Row Turning in Under-Canopy Robots
Field Robotics Engineering and Sciences Hub (FRESH), University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校野外机器人工程与科学中心（FRESH））；Department of Aerospace Engineering, University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校航空航天工程系）
University of Southern California（南加州大学）；Meta AI；KAIST（韩国科学技术院）；Georgia Tech（佐治亚理工学院）
We propose an approach for semantic imitation, which uses demonstrations from a source domain, e.g. human videos, to accelerate reinforcement learning (RL) in a different target domain, e.g. a robotic manipulator in a simulated kitchen. Instead of imitating low-level actions like joint velocities, our approach imitates the sequence of demonstrated semantic skills like "opening the microwave" or "turning on the stove". This allows us to transfer demonstrations across environments (e.g. real-world to simulated kitchen) and agent embodiments (e.g. bimanual human demonstration to robotic arm). We …
Billions of people use chopsticks, a simple yet versatile tool, for fine manipulation of everyday objects. The small, curved, and slippery tips of chopsticks pose a challenge for picking up small objects, making them a suitably complex test case. This paper leverages human demonstrations to develop an autonomous chopsticks-equipped robotic manipulator. Due to the lack of accurate models for fine manipulation, we explore model-free imitation learning, which traditionally suffers from the covariate shift phenomenon that causes poor generalization. We propose two approaches to reduce covariate sh…
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；National University of Defense Technology（国防科技大学）；University of Maryland, College Park（马里兰大学帕克分校）
We present a target-driven navigation system to improve mapless visual navigation in indoor scenes. Our method takes a multi-view observation of a robot and a target as inputs at each time step to provide a sequence of actions that move the robot to the target without relying on odometry or GPS at runtime. The system is learned by optimizing a combinational objective encompassing three key designs. First, we propose that an agent conceives the next observation before making an action decision. This is achieved by learning a variational generative module from expert demonstrations. We then prop…
University of California, Berkeley（加州大学伯克利分校）；Robotics at Google（谷歌机器人学）
Imitation learning is a popular approach for training visual navigation policies. However, collecting expert demonstrations for legged robots is challenging as these robots can be hard to control, move slowly, and cannot operate continuously for a long time. Here, we propose a zero-shot imitation learning approach for training a visual navigation policy on legged robots from human (third-person perspective) demonstrations, enabling high-quality navigation and cost-effective data collection. However, imitation learning from third-person demonstrations raises unique challenges. First, these demo…
Improving Robustness to Out-of-Distribution States in Imitation Learning via Deep Koopman-Boosted Diffusion Policy
Chair for Computer Aided Medical Procedures and Augmented Reality (CAMP), Technical University of Munich (TUM)（计算机辅助医疗程序与增强现实中心（CAMP），慕尼黑技术大学（TUM））；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Medical Intelligence and Robotic Cognition Lab (MIRoC), The University of Hong Kong (HKU)（医疗智能与机器人认知实验室（MIRoC），香港大学（HKU））
In this work, we introduce a new method for imitation learning from video demonstrations. Our method, Relational Mimic (RM), improves on previous visual imitation learning methods by combining generative adversarial networks and relational learning. RM is flexible and can be used in conjunction with other recent advances in generative adversarial imitation learning to better address the need for more robust and sample-efficient approaches. In addition, we introduce a new neural network architecture that improves upon the previous state-of-the-art in reinforcement learning and illustrate how in…
University of California, Berkeley（加州大学伯克利分校）；Facebook AI Research（Facebook人工智能研究院）；New York University（纽约大学）；Carnegie Mellon University（卡内基梅隆大学）
Visual imitation learning provides a framework for learning complex manipulation behaviors by leveraging human demonstrations. However, current interfaces for imitation such as kinesthetic teaching or teleoperation prohibitively restrict our ability to efficiently collect large-scale data in the wild. Obtaining such diverse demonstration data is paramount for the generalization of learned skills to novel scenarios. In this work, we present an alternate interface for imitation that simplifies the data collection process while allowing for easy transfer to robots. We use commercially available r…
