在监督学习中，近年来数据质量与数据整理的问题被能够摄取互联网规模数据的日益强大且富有表达力的模型所掩盖。然而，在机器人的离线学习中，我们根本没有互联网规模的数据，因此高质量数据集是必需品。在模仿学习（IL）这种利用专家演示进行机器人学习的样本高效范式中尤其如此。通过 IL 学到的策略在测试时会因动作预测中的误差累积而遭受状态分布偏移，这会导致策略无法从中恢复的未见过状态。与其设计新算法来解决分布偏移，另一种视角是开发评估和整理数据集的新方法。越来越多的证据表明，相同的 IL 算法在不同数据集上可能表现出显著不同的性能。这就需要一种形式化方法来定义“数据质量”指标，并进一步将其用于数据整理。在本工作中，我们通过分布偏移的视角，迈出了将模仿学习中的数据质量形式化的第一步：高质量数据集会促使策略在测试时保持在分布内。我们提出了塑造数据集质量的两个基本属性：i）动作散度：在某些状态下专家与所学策略之间的不匹配；以及 ii）转移多样性：在给定状态和动作下系统中存在的噪声。我们从理论上研究了模仿学习中这两个关键属性的联合效应，并对在多种不同数据源上训练的模型进行了实证分析。我们表明，状态多样性并不总是有益的，并展示了动作散度与转移多样性在实践中如何相互作用。
英文摘要：
In supervised learning, the question of data quality and curation has been over-shadowed in recent years by increasingly more powerful and expressive models that can ingest internet-scale data. However, in offline learning for robotics, we simply lack internet scale data, and so high quality datasets are a necessity. This is especially true in imitation learning (IL), a sample efficient paradigm for robot learning using expert demonstrations. Policies learned through IL suffer from state distribution shift at test time due to compounding errors in action prediction, which leads to unseen states that the policy cannot recover from. Instead of designing new algorithms to address distribution shift, an alternative perspective is to develop new ways of assessing and curating datasets. There is growing evidence that the same IL algorithms can have substantially different performance across different datasets. This calls for a formalism for defining metrics of "data quality" that can further be leveraged for data curation. In this work, we take the first step toward formalizing data quality for imitation learning through the lens of distribution shift: a high quality dataset encourages the policy to stay in distribution at test time. We propose two fundamental properties that shape the quality of a dataset: i) action divergence: the mismatch between the expert and learned policy at certain states; and ii) transition diversity: the noise present in the system for a given state and action. We investigate the combined effect of these two key properties in imitation learning theoretically, and we empirically analyze models trained on a variety of different data sources. We show that state diversity is not always beneficial, and we demonstrate how action divergence and transition diversity interact in practice.
Distortion-Resilient Robotic Imitation Learning for Autonomous Cable Routing
School of Artificial Intelligence, Harbin Institute of Technology（哈尔滨工业大学人工智能学院）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）；Pengcheng Laboratory（鹏城实验室）；Suzhou Research Institute, Harbin Institute of Technology（哈尔滨工业大学苏州研究院）
University of Tsinghua（清华大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Shanghai Qi Zhi Institute（上海期智研究院）；Hong Kong University of Science and Technology（香港科技大学）
本文提出策略对比模仿学习（PCIL），通过锚定不同策略学习对比表示空间并生成平滑余弦相似度奖励，以解决 AIL 判别器表示质量低的问题，在 DeepMind Control suite 上取得最先进性能。
MinInter: Minimizing Trajectory Interpolation During Data Augmentation for Imitation Learning
School of Automation and Intelligent Manufacturing, Southern University of Science and Technology（南方科技大学自动化与智能制造学院）；Pengcheng Laboratory（鹏城实验室）；Carnegie Mellon University（卡内基梅隆大学）
Translating Flow to Policy via Hindsight Online Imitation
Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）；University of California San Diego（加州大学圣地亚哥分校）；Washington University in St. Louis（圣路易斯华盛顿大学）；Shanghai Jiao Tong University（上海交通大学）；Shanghai Qi Zhi Institute（上海启智研究所）
Training-Free Imitation Learning with Closed-Form Diffusion Policies
Australian Center for Robotics, ARIAM Hub, and School of Aerospace, Mechanical and Mechatronic Engineering University of Sydney（澳大利亚机器人中心、ARIAM中心和悉尼大学航空航天、机械与机电工程学院）
Federated Imitation Learning: A Privacy Considered Imitation Learning Framework for Cloud Robotic Systems with Heterogeneous Sensor Data
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Hong Kong University of Science and Technology（香港科技大学）
Robust Offline Imitation Learning Through State-level Trajectory Stitching
National Key Lab of Autonomous Intelligent Unmanned Systems, Beijing Institute of Technology, Beijing 100081, China（自主智能无人系统国家重点实验室，北京理工大学，北京100081，中国）；Department of Control Science and Engineering, Harbin Institute of Technology, Harbin 150001, China（控制科学与工程学院，哈尔滨工业大学，哈尔滨150001，中国）
Neural-Primitive: An Efficient End-to-end Local Planner with Primitive-based Imitation Learning for Autonomous Flight
Institute of Cyber-Systems and Control, College of Control Science and Engineering, Zhejiang University（浙江大学控制科学与工程学院 Cyber-系统与控制研究所）；Huzhou Institute, Zhejiang University（浙江大学湖州研究院）；Institute of Systems Engineering, China Academy of Engineering Physics（中国工程物理研究院系统工程研究所）；School of Automation, Hangzhou Dianzi University（杭州电子科技大学自动化学院）；Department of Automation, North China Electric Power University (Baoding)（华北电力大学（保定）自动化系）；Differential Robotics Technology Company（差分机器人技术公司）
Semi-Supervised Imitation Learning with Mixed Qualities of Demonstrations for Autonomous Driving
Seoul National University（首尔国立大学）；Dongguk University（东国大学）；Department of Electrical and Computer Engineering（电子与计算机工程系）；ASRI（ASRI（人工智能研究中心））；School of Computer Science and Engineering（计算机科学与工程学院）
Self-evolved Imitation Learning in Simulated World
College of Science and Engineering, Hamad Bin Khalifa University（1 科学与工程学院，哈马德·本·卡西姆大学）；College of Computer Science and Technology, Zhejiang University（2 计算机科学与技术学院，浙江大学）；Guangdong Provincial Key Laboratory of Sensing Physics and System Integration Applications（4 广东省传感物理与系统集成应用重点实验室）
A Wearable Robotic Hand for Hand-over-Hand Imitation Learning
Tsinghua-Berkeley Shenzhen Institute, Tsinghua University（清华大学清华-伯克利深圳学院）；Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）；Shanghai AI Lab（上海人工智能实验室）；Shanghai Qi Zhi Institute（上海期智研究院）
Easy-IIL: Reducing Human Operational Burden in Interactive Imitation Learning via an Assistant Expert
Shenzhen Key Laboratory of Robotics and Computer Vision（深圳机器人与计算机视觉重点实验室）；Department of Electronic and Electrical Engineering（电子与电气工程系）；Department of Robotics, Perception, and Learning（机器人、感知与学习系）
Symbolic Imitation Learning: From Black-Box to Explainable Driving Policies
Department of Science and Engineering, the George Washington University（科学与工程系，乔治·华盛顿大学）；Autonomous Vehicles Lab, Department of Mechanical Engineering, University of Surrey（自主车辆实验室，机械工程系，萨里大学）
Robotic Manipulation via Imitation Learning: Taxonomy, Evolution, Benchmark, and Challenges
Ecole Centrale de Lyon, CNRS, Universite Claude Bernard Lyon 1, INSA Lyon, Université Lumière Lyon 2, LIRIS, UMR5205（Lyon中央理工学院、CNRS、Claude Bernard Lyon 1大学、INSA Lyon、Lyon大学、LIRIS、UMR5205）；Beihang University（北航大学）；Dalian University of Technology（大连理工大学）；Institut Universitaire de France (IUF)（法国大学研究院（IUF））
Can a Robot Read Braille? - Learning to Adapt Contact via Imitation Learning for Tactile Braille Recognition
Shenzhen University（深圳大学）；Great Bay University（大湾区大学）；China Pharmaceutical University（中国药科大学）；The University of Hong Kong（香港大学）；Tsinghua University（清华大学）；École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）；University of Belgrade（贝尔格莱德大学）
