Imitation learning practitioners have often noted that conditioning policies on previous actions leads to a dramatic divergence between "held out" error and performance of the learner in situ. Interactive approaches can provably address this divergence but require repeated querying of a demonstrator. Recent work identifies this divergence as stemming from a "causal confound" in predicting the current action, and seek to ablate causal aspects of current state using tools from causal inference. In this work, we argue instead that this divergence is simply another manifestation of covariate shift, exacerbated particularly by settings of feedback between decisions and input features. The learner often comes to rely on features that are strongly predictive of decisions, but are subject to strong covariate shift.
Our work demonstrates a broad class of problems where this shift can be mitigated, both theoretically and practically, by taking advantage of a simulator but without any further querying of expert demonstration. We analyze existing benchmarks used to test imitation learning approaches and find that these benchmarks are realizable and simple and thus insufficient for capturing the harder regimes of error compounding seen in real-world decision making problems. We find, in a surprising contrast with previous literature, but consistent with our theory, that naive behavioral cloning provides excellent results. We detail the need for new standardized benchmarks that capture the phenomena seen in robotics problems.
Counter-example guided Imitation Learning of Feedback Controllers from Temporal Logic Specifications
Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔理工学院）；VERIMAG（VERIMAG 实验室）；Decyphir SAS（Decyphir SAS 公司）；IIT Kanpur（印度理工学院坎普尔分校）；Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）
Feasibility-aware Imitation Learning from Observation with Multimodal Feedback
Division of Information Science, Graduate School of Science and Technology, Nara Institute of Science and Technology (NAIST)（信息科学系，科学技术研究生院，科学与技术国立研究所（NAIST））
Agile Autonomous Driving using End-to-End Deep Imitation Learning
Institute for Robotics and Intelligent Machines（机器人与智能机器研究所）；School of Electrical and Computer Engineering（电气与计算机工程学院）；Georgia Institute of Technology（佐治亚理工学院）
Adaptive Wiping: Adaptive contact-rich manipulation through few-shot imitation learning with Force-Torque feedback and pre-trained object representations
Department of Mechano-Informatics, The University of Tokyo（东京大学机械信息学系）；National Institute of Advanced Industrial Science and Technology（国家先进工业科学与技术研究院）；Department of Mechanical Systems Engineering, Tokyo University of Agriculture and Technology（东京农业大学机械系统工程系）；Department of Mechanical Engineering, The University of Tokyo（东京大学机械工程系）
Conformalized Interactive Imitation Learning: Handling Expert Shift and Intermittent Feedback
Robotics Institute, School of Computer Science, Carnegie Mellon University（机器人研究所、计算机科学学院、卡内基梅隆大学）；Departments of Statistics and Machine Learning, Carnegie Mellon University（统计学与机器学习系、卡内基梅隆大学）
A Wearable Robotic Hand for Hand-over-Hand Imitation Learning
Tsinghua-Berkeley Shenzhen Institute, Tsinghua University（清华大学清华-伯克利深圳学院）；Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）；Shanghai AI Lab（上海人工智能实验室）；Shanghai Qi Zhi Institute（上海期智研究院）
Robust Offline Imitation Learning Through State-level Trajectory Stitching
National Key Lab of Autonomous Intelligent Unmanned Systems, Beijing Institute of Technology, Beijing 100081, China（自主智能无人系统国家重点实验室，北京理工大学，北京100081，中国）；Department of Control Science and Engineering, Harbin Institute of Technology, Harbin 150001, China（控制科学与工程学院，哈尔滨工业大学，哈尔滨150001，中国）
VFILC: Accurate Frequency Extrapolations in Imitation Learning via Sampling Frequency ILC
Grad. School of Science；Technology University of Tsukuba Tsukuba, Japan；Engineering Saitama University Saitama, Japan；Information Engineering University of Tsukuba Tsukuba, Japan
Sample-Efficient Expert Query Control in Active Imitation Learning via Conformal Prediction
Department of Computer Science, University of Southern California（计算机科学系，南加州大学）；Department of Information Technology and Electrical Engineering, ETH Zurich（信息科技与电气工程系，苏黎世联邦理工学院）
Force Generative Imitation Learning: Bridging Position Trajectory and Force Commands through Control Technique
Graduate School of Systems and Information Engineering, University of Tsukuba（东京大学系统与信息工程研究生院）；Graduate School of Science and Engineering, Saitama University（埼玉大学科学与工程研究生院）；Department of Intelligent Interaction Technologies, Institute of Systems and Information Engineering, University of Tsukuba（东京大学系统与信息工程研究所智能交互技术系）
Tube Diffusion Policy: Reactive Visual-Tactile Policy Learning for Contact-rich Manipulation
Meta Reality Labs Research（Meta现实实验室）；Idiap Research Institute（Idiap研究机构）；École Polytechnique Fédérale de Lausanne（瑞士联邦理工学院）；University of Toronto（多伦多大学）；Purdue University（普渡大学）
Difference-Aware Retrieval Policies for Imitation Learning
Paul G. Allen School of Computer Science & Engineering, University of Washington（华盛顿大学保罗·G·艾伦计算机科学与工程学院）；Toyota Research Institute（丰田研究所）；Google DeepMind（谷歌DeepMind）；Mila
