Recent work in visual end-to-end learning for robotics has shown the promise of imitation learning across a variety of tasks. Such approaches are expensive both because they require large amounts of real world training demonstrations and because identifying the best model to deploy in the real world requires time-consuming real-world evaluations. These challenges can be mitigated by simulation: by supplementing real world data with simulated demonstrations and using simulated evaluations to identify high performing policies. However, this introduces the well-known "reality gap" problem, where simulator inaccuracies decorrelate performance in simulation from that of reality. In this paper, we build on top of prior work in GAN-based domain adaptation and introduce the notion of a Task Consistency Loss (TCL), a self-supervised loss that encourages sim and real alignment both at the feature and action-prediction levels. We demonstrate the effectiveness of our approach by teaching a mobile manipulator to autonomously approach a door, turn the handle to open the door, and enter the room. The policy performs control from RGB and depth images and generalizes to doors not encountered in training data. We achieve 72% success across sixteen seen and unseen scenes using only ~16.2 hours of teleoperated demonstrations in sim and real. To the best of our knowledge, this is the first work to tackle latched door opening from a purely end-to-end learning approach, where the task of navigation and manipulation are jointly modeled by a single neural network.
BiKC+: Bimanual Hierarchical Imitation with Keypose-Conditioned Coordination-Aware Consistency Policies
Centre for Transformative Garment Production, The University of Hong Kong（变革性服装生产中心，香港大学）；School of Computing and Data Science, The University of Hong Kong（计算与数据科学学院，香港大学）；Advanced Manufacturing Lab, Huawei Technologies（先进制造实验室，华为技术）
Imitation Learning Policy based on Multi-Step Consistent Integration Shortcut Model
State Key Laboratory of Robotics and Systems, Harbin Institute of Technology（机器人与系统国家重点实验室，哈尔滨工业大学）；School of Computer Science and Engineering, Tianjin University of Technology（计算机科学与工程学院，天津理工大学）
Tsinghua University（清华大学）；Harbin Institute of Technology（哈尔滨工业大学）；Jiangnan University（江南大学）；Zhejiang University（浙江大学）；Tongji University（同济大学）；University of Pennsylvania（宾夕法尼亚大学）
University of Tokyo（东京大学）；Technische Universität Darmstadt（达姆施塔特工业大学）；University of Lincoln（林肯大学）；Carnegie Mellon University（卡内基梅隆大学）；University of California, Berkeley（加州大学伯克利分校）
VILP: Imitation Learning with Latent Video Planning
School of Industrial Engineering, Purdue University（普渡大学工业工程学院）；Elmore Family School of Electrical and Computer Engineering, Purdue University（普渡大学埃尔莫尔家族电气与计算机工程学院）
Compact Task-Aligned Imitation Learning for Laboratory Automation
Spatial Robotics Research Center, Fujitsu Limited.（富士通株式会社空间机器人研究中心）；Faculty of Science and Engineering, Waseda University（早稻田大学理工学部）；National Institute of Advanced Industrial Science and Technology（国家工业科学与技术研究院）
Rewind-IL: Online Failure Detection and State Respawning for Imitation Learning
College of Connected Computing, Vanderbilt University（范德比尔特大学连接计算学院）；School of Computer Science, University of Waterloo（滑铁卢大学计算机科学学院）；School of Computer Science, The University of Sydney（悉尼大学计算机科学学院）；Australian Centre for Robotics, The University of Sydney（悉尼大学机器人研究中心）
Long-Horizon Visual Imitation Learning via Plan and Code Reflection
Beijing Key Laboratory of Intelligent Information Technology, School of Computer Science and Technology, Beijing Institute of Technology, China（北京智能信息科技重点实验室，计算机科学与技术学院，北京理工大学，中国）；Guangdong Laboratory of Machine Perception and Intelligent Computing, Shenzhen MSU-BIT University, China（广东机器感知与智能计算实验室，深圳MSU-BIT大学，中国）；Beijing Innovation Center of Humanoid Robotics, China（北京人形机器人创新中心，中国）
Gaze-Guided Task Decomposition for Imitation Learning in Robotic Manipulation
The University of Tokyo（东京大学）；Graduate School of Information Science and Technology（信息科学与技术研究生院）；Laboratory for Intelligent Systems and Informatics（智能系统与信息学实验室）
Learn by Observation: Imitation Learning for Drone Patrolling from Videos of A Human Navigator
School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）；Laboratory for Computational Sensing and Robotics, Johns Hopkins University（约翰斯·霍普金斯大学计算感知与机器人实验室）
Where Small Mistakes Matter: Quantifying State Criticality for Imitation Learning
University of Southern California（南加州大学）；Stanford University（斯坦福大学）；Georgia Institute of Technology（佐治亚理工学院）；Inria Paris（巴黎信息与自动化研究所）；PSL University（巴黎文理研究大学）
Towards Diverse Behaviors: A Benchmark for Imitation Learning with Human Demonstrations
Autonomous Learning Robots, Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院自主学习机器人研究所）；Intuitive Robots Lab, Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院直觉机器人实验室）
On the Generalization Capabilities, Design Choices and Limitations of Keypoint Imitation Learning
AI and Robotics Lab, IDLAB-AIRO, Ghent University-imec（人工智能与机器人实验室，IDLAB-AIRO，根特大学-imec）；Robotics, Perception and Learning Lab, (RPL), EECS, KTH Royal Institute of Technology（机器人、感知与学习实验室，（RPL），EECS，皇家理工学院）；INCAR Robotics AB, Stockholm, Sweden（INCAR机器人公司，斯德哥尔摩，瑞典）
Restoring Noisy Demonstration for Imitation Learning With Diffusion Models
Graduate Institute of Communication Engineering, National Taiwan University（国立台湾大学通信工程研究所）；Data Science Degree Program, National Taiwan University and Academia Sinica（国立台湾大学数据科学学士学位计划）；Department of Electrical Engineering, National Taiwan University（国立台湾大学电子工程系）
Imitation and Adaptation Based on Consistency: A Quadruped Robot Imitates Animals from Videos Using Deep Reinforcement Learning
Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences（中国科学院机器人与智能制造创新研究院）；Westlake University（西湖大学）；University of Chinese Academy of Sciences（中国科学院大学）
ImitDiff: Transferring Foundation-Model Priors for Distraction Robust Visuomotor Policy
Ningbo Global Innovation Center, Zhejiang University（浙江大学宁波全球创新中心）；Tsinghua University（清华大学）；Youtu, Tencent（腾讯优图）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；College of Control Science and Engineering, Zhejiang University（浙江大学控制科学与工程学院）
KOI: Accelerating Online Imitation Learning via Hybrid Key-state Guidance
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Northwestern Polytechnical University（西北工业大学）；Institute of Artificial Intelligence, China Telecom Corp Ltd（中国电信股份有限公司人工智能研究院）
