Robot learning holds the promise of learning policies that generalize broadly. However, such generalization requires sufficiently diverse datasets of the task of interest, which can be prohibitively expensive to collect. In other fields, such as computer vision, it is common to utilize shared, reusable datasets, such as ImageNet, to overcome this challenge, but this has proven difficult in robotics. In this paper, we ask: what would it take to enable practical data reuse in robotics for end-to-end skill learning? We hypothesize that the key is to use datasets with multiple tasks and multiple domains, such that a new user that wants to train their robot to perform a new task in a new domain can include this dataset in their training process and benefit from cross-task and cross-domain generalization. To evaluate this hypothesis, we collect a large multi-domain and multi-task dataset, with 7,200 demonstrations constituting 71 tasks across 10 environments, and empirically study how this data can improve the learning of new tasks in new environments. We find that jointly training with the proposed dataset and 50 demonstrations of a never-before-seen task in a new domain on average leads to a 2x improvement in success rate compared to using target domain data alone. We also find that data for only a few tasks in a new domain can bridge the domain gap and make it possible for a robot to perform a variety of prior tasks that were only seen in other domains. These results suggest that reusing diverse multi-task and multi-domain datasets, including our open-source dataset, may pave the way for broader robot generalization, eliminating the need to re-collect data for each new robot learning project.
Neuro-Symbolic Imitation Learning: Discovering Symbolic Abstractions for Skill Learning
Intelligent Autonomous Systems, TU Darmstadt, Germany（图腾达姆施塔特大学智能自主系统研究所）；German Research Center for AI, Germany（德国人工智能研究中心）；Hessian Centre for Artificial Intelligence, Germany（黑森州人工智能中心）；Honda Research Institute EU, Germany（本田欧洲研究院）
Traj2Action: A Co-Denoising Framework for Trajectory-Guided Human-to-Robot Skill Transfer
MAPLE Lab, Westlake University（西湖大学MAPLE实验室）；Zhejiang University（浙江大学）；Huazhong University of Science and Technology（华中科技大学）；Institute of Advanced Technology, Westlake Institute for Advanced Study（西湖大学先进技术研究所）
CLASP: Language-Driven Robot Skill Selection and Composition using Task-Parameterized Learning
German Aerospace Center (DLR), Institute of Robotics and Mechatronics (RMC)（德国航空航天中心（DLR），机器人与机电一体化研究所（RMC））；Technical University of Munich (TUM)（慕尼黑工业大学（TUM））
GROVE: A Generalized Reward for Learning Open-Vocabulary Physical Skill
Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）；State Key Laboratory of General Artificial Intelligence, BIGAI（北京通用人工智能研究院 通用人工智能国家重点实验室）；School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）；Department of Automation, Tsinghua University（清华大学自动化系）
CoINS: Counterfactual Interactive Navigation via Skill-Aware VLM
School of Advanced Manufacturing and Robotics, Peking University（北京大学先进制造与机器人学院）；Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong（香港中文大学机械与自动化工程系）；College of Design and Engineering, National University Of Singapore（新加坡国立大学设计与工程学院）；Department of Computer Science, The University of Manchester（曼彻斯特大学计算机科学系）
