RESTA: Time-Scaling Augmentation for Data-Efficient Robot Imitation Learning
Global College, Shanghai Jiao Tong University（上海交通大学Global College）；Shanghai Institute of Physical Intelligence and Robotics (PAIR)（上海物理智能与机器人研究所（PAIR））；Duke Kunshan University（昆山杜克大学）
Yunpeng Jiang, Jianshu Hu, Han Fang, Zhiyuan Guan, Paul Weng, Yutong Ban
通过模仿学习训练的机器人操作策略通常依赖于在单一任务方向和单一时间分辨率下记录的高成本演示数据。我们提出了基于反转与步长的轨迹增强方法（Reversal and Stride-based Trajectory Augmentation，RESTA），该方法利用时间缩放数据增强技术，通过轨迹反转和时间重采样提升数据效率。轨迹反转增强会反转并重新标记具有相反目标的配对任务之间的演示数据，无需额外环境交互即可复用已记录的 rollout 数据。由于接触丰富的操作无法完全反转，我们将反转视为近似的配对任务先验而非物理对称性。时间步长增强会改变观测和动作的采样步长，并让策略基于由此产生的偏移量进行条件化，支持固定或随机时间调度。在16项配对双臂操作任务上开展的实验表明，轨迹反转在低数据 regime 中效果尤为显著，而时间步长增强可提升随机时间调度下的任务成功率。在每个任务仅收集25和50个独立演示的情况下，RESTA分别达到78.4%和85.2%的平均成功率，与使用两倍数量演示训练的基线方法（分别为77.3%和85.4%）相当或更优。这些结果表明，RESTA能够提升机器人模仿学习的数据效率。
英文摘要：
Robot manipulation policies trained through imitation learning commonly rely on costly demonstrations recorded in one task direction and at one temporal resolution. We propose Reversal and Stride-based Trajectory Augmentation (RESTA), which exploits time-scaling data augmentation to improve data efficiency through trajectory reversal and temporal resampling. Trajectory-reversal augmentation reverses and relabels demonstrations between paired tasks with opposite objectives, reusing recorded rollouts without additional environment interaction. Because contact-rich manipulation is not exactly reversible, we treat reversal as an approximate paired-task prior rather than a physical symmetry. Temporal-stride augmentation varies observation and action sampling strides and conditions the policy on the resulting offsets, enabling fixed or randomized temporal schedules. Experiments across 16 paired bimanual manipulation tasks show that trajectory reversal is particularly effective in the low-data regime and that temporal-stride augmentation improves success under randomized temporal schedules. With 25 and 50 independently collected demonstrations per task, RESTA achieves 78.4% and 85.2% average success, respectively, matching or exceeding baselines trained with twice as many demonstrations (77.3% and 85.4%). These results show that RESTA can improve data efficiency in robot imitation learning.
Khoury College of Computer Sciences（Khoury计算机科学学院）；Northeastern University（东北大学）
In robotic manipulation, acquiring samples is extremely expensive because it often requires interacting with the real world. Traditional image-level data augmentation has shown the potential to improve sample efficiency in various machine learning tasks. However, image-level data augmentation is insufficient for an imitation learning agent to learn good manipulation policies in a reasonable amount of demonstrations. We propose Simulation-augmented Equivariant Imitation Learning (SEIL), a method that combines a novel data augmentation strategy of supplementing expert trajectories with simulated…
MinInter: Minimizing Trajectory Interpolation During Data Augmentation for Imitation Learning
School of Automation and Intelligent Manufacturing, Southern University of Science and Technology（南方科技大学自动化与智能制造学院）；Pengcheng Laboratory（鹏城实验室）；Carnegie Mellon University（卡内基梅隆大学）
Variable-Speed Teaching-Playback as Real-World Data Augmentation for Imitation Learning
Master’s Program in Intelligent and Mechanical Interaction Systems, Degree Programs in Systems and Information and Engineering, Graduate School of Science and Technology, University of Tsukuba, Japan（茨川大学科学技术研究生院系统与信息工程学系智能与机械交互系统硕士课程）；Doctoral Program in Intelligent and Mechanical Interaction Systems, Degree Programs in Systems and Information and Engineering, Graduate School of Science and Technology, University of Tsukuba, Japan（茨川大学科学技术研究生院系统与信息工程学系智能与机械交互系统博士课程）；Department of Intelligent Interaction Technologies, Institute of Systems and Information Engineering, University of Tsukuba, Japan（茨川大学系统与信息工程研究所智能交互技术系）；Graduate School of Science and Engineering, Saitama University, Saitama, Japan（埼玉大学工学研究科）
In this paper we present a complete study of an end-to-end imitation learning system for speed control of a real car, based on a neural network with a Long Short Term Memory (LSTM). To achieve robustness and generalization from expert demonstrations, we propose data augmentation and label augmentation that are relevant for imitation learning in longitudinal control context. Based on front camera image only, our system is able to correctly control the speed of a car in simulation environment, and in a real car on a challenging test track. The system also shows promising results in open road con…
Imitation learning is a promising approach to end-to-end training of autonomous vehicle controllers. Typically the driving process with such approaches is entirely automatic and black-box, although in practice it is desirable to control the vehicle through high-level commands, such as telling it which way to go at an intersection. In existing work this has been accomplished by the application of a branched neural architecture, since directly providing the command as an additional input to the controller often results in the command being ignored. In this work we overcome this limitation by lea…
Nara Institute of Science and Technology（奈良先端科学技术大学院大学）
Robust imitation learning using disturbance injections overcomes issues of limited variation in demonstrations. However, these methods assume demonstrations are optimal, and that policy stabilization can be learned via simple augmentations. In real-world scenarios, demonstrations are often of diverse-quality, and disturbance injection instead learns sub-optimal policies that fail to replicate desired behavior. To address this issue, this paper proposes a novel imitation learning framework that combines both policy robustification and optimal demonstration learning. Specifically, this combinato…
RoboManipBaselines: A Unified Framework for Imitation Learning in Robotic Manipulation across Real and Simulation Environments
Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology (AIST)（日本国家先进工业科学技术研究院人工智能研究中心）；CNRS-AIST JRL (Joint Robotics Laboratory), IRL（法国国家科学研究中心与日本AIST联合机器人实验室）；Institute for AI and Robotics, Future Robotics Organization, Waseda University（早稻田大学未来机器人组织人工智能与机器人研究所）；AI Robot Association (AIRoA)（人工智能机器人协会）
City University of Hong Kong（香港城市大学）；The Chinese University of Hong Kong（香港中文大学）
Automatic laparoscope motion control is fundamentally important for surgeons to efficiently perform operations. However, its traditional control methods based on tool tracking without considering information hidden in surgical scenes are not intelligent enough, while the latest supervised imitation learning (IL)-based methods require expensive sensor data and suffer from distribution mismatch issues caused by limited demonstrations. In this paper, we propose a novel Imitation Learning framework for Laparoscope Control (ILLC) with reinforcement learning (RL), which can efficiently learn the con…
Training-Free Imitation Learning with Closed-Form Diffusion Policies
Australian Center for Robotics, ARIAM Hub, and School of Aerospace, Mechanical and Mechatronic Engineering University of Sydney（澳大利亚机器人中心、ARIAM中心和悉尼大学航空航天、机械与机电工程学院）
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；IIIS, Tsinghua（清华大学智能产业研究院）；NVIDIA Research（英伟达研究院）
Imitation learning offers a promising path for robots to learn general-purpose behaviors, but traditionally has exhibited limited scalability due to high data supervision requirements and brittle generalization. Inspired by recent advances in multi-task imitation learning, we investigate the use of prior data from previous tasks to facilitate learning novel tasks in a robust, data-efficient manner. To make effective use of the prior data, the robot must internalize knowledge from past experiences and contextualize this knowledge in novel tasks. To that end, we develop a skill-based imitation l…
RoboEngine: Plug-and-Play Robot Data Augmentation with Semantic Robot Segmentation and Background Generation
Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）；Shanghai Qi Zhi Institute（上海启智研究院）
Columbia University（哥伦比亚大学）；Meta AI；Carnegie Mellon University（卡内基梅隆大学）
Large-scale training have propelled significant progress in various sub-fields of AI such as computer vision and natural language processing. However, building robot learning systems at a comparable scale remains challenging. To develop robots that can perform a wide range of skills and adapt to new scenarios, efficient methods for collecting vast and diverse amounts of data on physical robot systems are required, as well as the capability to train high-capacity policies using such datasets. In this work, we propose a framework for scaling robot learning, with specific focus on multi-task and …
We present a learning-based planner that aims to robustly drive a vehicle by mimicking human drivers' driving behavior. We leverage a mid-to-mid approach that allows us to manipulate the input to our imitation learning network freely. With that in mind, we propose a novel feedback synthesizer for data augmentation. It allows our agent to gain more driving experience in various previously unseen environments that are likely to encounter, thus improving overall performance. This is in contrast to prior works that rely purely on random synthesizers. Furthermore, rather than completely commit to i…
Generative adversarial imitation learning (GAIL) has shown promising results by taking advantage of generative adversarial nets, especially in the field of robot learning. However, the requirement of isolated single modal demonstrations limits the scalability of the approach to real world scenarios such as autonomous vehicles' demand for a proper understanding of human drivers' behavior. In this paper, we propose a novel multi-modal GAIL framework, named Triple-GAIL, that is able to learn skill selection and imitation jointly from both expert demonstrations and continuously generated experienc…
University of California, Berkeley（加州大学伯克利分校）；Facebook AI Research（Facebook人工智能研究院）；New York University（纽约大学）；Carnegie Mellon University（卡内基梅隆大学）
Visual imitation learning provides a framework for learning complex manipulation behaviors by leveraging human demonstrations. However, current interfaces for imitation such as kinesthetic teaching or teleoperation prohibitively restrict our ability to efficiently collect large-scale data in the wild. Obtaining such diverse demonstration data is paramount for the generalization of learned skills to novel scenarios. In this work, we present an alternate interface for imitation that simplifies the data collection process while allowing for easy transfer to robots. We use commercially available r…
Rewind-IL: Online Failure Detection and State Respawning for Imitation Learning
College of Connected Computing, Vanderbilt University（范德比尔特大学连接计算学院）；School of Computer Science, University of Waterloo（滑铁卢大学计算机科学学院）；School of Computer Science, The University of Sydney（悉尼大学计算机科学学院）；Australian Centre for Robotics, The University of Sydney（悉尼大学机器人研究中心）
Improving Imitation Learning Efficiency for Manipulation through Geometric Prior Pretraining
Waseda University（早稻田大学）；National Institute of Advanced Industrial Science and Technology (AIST)（产业技术综合研究所（AIST））；Waseda Research Institute for Science and Engineering（早稻田大学理工学研究所）；University of Oxford（牛津大学）
In this work, we present a lightweight pipeline for robust behavioral cloning of a human driver using end-to-end imitation learning. The proposed pipeline was employed to train and deploy three distinct driving behavior models onto a simulated vehicle. The training phase comprised of data collection, balancing, augmentation, preprocessing and training a neural network, following which, the trained model was deployed onto the ego vehicle to predict steering commands based on the feed from an onboard camera. A novel coupled control law was formulated to generate longitudinal control commands on-…
Self-evolved Imitation Learning in Simulated World
College of Science and Engineering, Hamad Bin Khalifa University（1 科学与工程学院，哈马德·本·卡西姆大学）；College of Computer Science and Technology, Zhejiang University（2 计算机科学与技术学院，浙江大学）；Guangdong Provincial Key Laboratory of Sensing Physics and System Integration Applications（4 广东省传感物理与系统集成应用重点实验室）
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Army Research Laboratory（陆军研究实验室）；Google（谷歌）
Social navigation is the capability of an autonomous agent, such as a robot, to navigate in a 'socially compliant' manner in the presence of other intelligent agents such as humans. With the emergence of autonomously navigating mobile robots in human populated environments (e.g., domestic service robots in homes and restaurants and food delivery robots on public sidewalks), incorporating socially compliant navigation behaviors on these robots becomes critical to ensuring safe and comfortable human robot coexistence. To address this challenge, imitation learning is a promising framework, since …
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
In this paper, we analyze the behavior of existing techniques and design new solutions for the problem of one-shot visual imitation. In this setting, an agent must solve a novel instance of a novel task given just a single visual demonstration. Our analysis reveals that current methods fall short because of three errors: the DAgger problem arising from purely offline training, last centimeter errors in interacting with objects, and mis-fitting to the task context rather than to the actual task. This motivates the design of our modular approach where we a) separate out task inference (what to d…
University of Toronto Robotics Institute（多伦多大学机器人研究所）
The robustness of visual navigation policies trained through imitation often hinges on the augmentation of the training image-action pairs. Traditionally, this has been done by collecting data from multiple cameras, by using standard data augmentations from computer vision, such as adding random noise to each image, or by synthesizing training images. In this paper we show that there is another practical alternative for data augmentation for visual navigation based on extrapolating viewpoint embeddings and actions nearby the ones observed in the training data. Our method makes use of the geome…
DTCCL: Disengagement-Triggered Contrastive Continual Learning for Autonomous Bus Planners
School of Vehicle and Mobility, Tsinghua University（车辆与移动系统学院，清华大学）；Dongfeng Motor Corporation Research and Development General Institute（东风汽车集团研发总院）；Dongfeng Yuexiang Technology Co., Ltd.（东风悦铃科技有限公司）
Kyushu Institute of Technology（九州工业大学）；RIKEN center for Advanced Intelligence Project（理化学研究所先进智能项目中心）
Imitation learning is an intuitive approach for teaching motion to robotic systems. Although previous studies have proposed various methods to model demonstrated movement primitives, one of the limitations of existing methods is that the shape of the trajectories are encoded in high dimensional space. The high dimensionality of the trajectory representation can be a bottleneck in the subsequent process such as planning a sequence of primitive motions. We address this problem by learning the latent space of the robot trajectory. If the latent variable of the trajectories can be learned, it can …
We cast visual imitation as a visual correspondence problem. Our robotic agent is rewarded when its actions result in better matching of relative spatial configurations for corresponding visual entities detected in its workspace and teacher's demonstration. We build upon recent advances in Computer Vision,such as human finger keypoint detectors, object detectors trained on-the-fly with synthetic augmentations, and point detectors supervised by viewpoint changes and learn multiple visual entity detectors for each demonstration without human annotations or robot interactions. We empirically show…
Expert demonstrations are a rich source of supervision for training visual robotic manipulation policies, but imitation learning methods often require either a large number of demonstrations or expensive online expert supervision to learn reactive closed-loop behaviors. In this work, we introduce SPARTN (Synthetic Perturbations for Augmenting Robot Trajectories via NeRF): a fully-offline data augmentation scheme for improving robot policies that use eye-in-hand cameras. Our approach leverages neural radiance fields (NeRFs) to synthetically inject corrective noise into visual demonstrations, us…
WorldSample: Closed-loop Real-robot RL with World Modelling
PINE Lab, School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore（南洋理工大学电气与电子工程学院PINE实验室）；Department of Electronic Engineering, Tsinghua University, Beijing, China（清华大学电子工程系）；School of Automation, Central South University, Changsha, China（中南大学自动化学院）；School of Automation, Beijing University of Posts and Telecommunications, Beijing, China（北京邮电大学自动化学院）
Tsinghua University（清华大学）；University of Pennsylvania（宾夕法尼亚大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；Shanghai Qi Zhi Institute（上海期智研究院）
Across applications spanning supervised classification and sequential control, deep learning has been reported to find "shortcut" solutions that fail catastrophically under minor changes in the data distribution. In this paper, we show empirically that DNNs can be coaxed to avoid poor shortcuts by providing an additional "priming" feature computed from key input features, usually a coarse output estimate. Priming relies on approximate domain knowledge of these task-relevant key input features, which is often easy to obtain in practical settings. For example, one might prioritize recent frames …
MIT Department of Aeronautics and Astronautics（麻省理工学院航空航天系）
We propose a demonstration-efficient strategy to compress a computationally expensive Model Predictive Controller (MPC) into a more computationally efficient representation based on a deep neural network and Imitation Learning (IL). By generating a Robust Tube variant (RTMPC) of the MPC and leveraging properties from the tube, we introduce a data augmentation method that enables high demonstration-efficiency, being capable to compensate the distribution shifts typically encountered in IL. Our approach opens the possibility of zero-shot transfer from a single demonstration collected in a nomina…
FoAM: Foresight-Augmented Multi-Task Imitation Policy for Robotic Manipulation
University of Southern California（南加州大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, Shanghai Jiao Tong University（上海交通大学人工智能学院）
