RH20T-P: A Primitive-Level Robotic Dataset Towards Composable Generalization Agents
Shanghai AI Laboratory(上海人工智能实验室);School of Software, Beihang University(北京航空航天大学软件学院);Shanghai Jiao Tong University(上海交通大学);University of Sydney(悉尼大学)
Achieving generalizability in solving out-of-distribution tasks is one of the ultimate goals of learning robotic manipulation. Recent progress of Vision-Language Models (VLMs) has shown that VLM-based task planners can alleviate the difficulty of solving novel tasks, by decomposing the compounded tasks as a plan of sequentially executing primitive-level skills that have been already mastered. It is also promising for robotic manipulation to adapt such composable generalization ability, in the form of composable generalization agents (CGAs). However, the community lacks of reliable design of primitive skills and a sufficient amount of primitive-level data annotations. Therefore, we propose RH20T-P, a primitive-level robotic manipulation dataset, which contains about 38k video clips covering 67 diverse manipulation tasks in real-world scenarios. Each clip is manually annotated according to a set of meticulously designed primitive skills that are common in robotic manipulation. Furthermore, we standardize a plan-execute CGA paradigm and implement an exemplar baseline called RA-P on our RH20T-P, whose positive performance on solving unseen tasks validates that the proposed dataset can offer composable generalization ability to robotic manipulation agents.
PrimitiveVLA: Learning Reusable Motion Primitives for Efficient and Generalizable Robotic Manipulation
State Key Lab of Processors, Institute of Computing Technology, CAS(处理器国家重点实验室,计算技术研究所,中国科学院);Jiangsu Key Laboratory of AI for Industries, Institute of AI for Industries, CAS(江苏人工智能工业重点实验室,人工智能工业研究所,中国科学院);University of Chinese Academy of Sciences(中国科学院大学);Cambricon Technologies(寒武科技);Intelligent Software Research Center, Institute of Software, CAS(软件研究所智能软件研究中心,中国科学院);University of Science and Technology of China(中国科学技术大学)
Adaptive Vision-Language Grasping via Composable Foundation Priors and Generalizable Grasp Synthesis
School of Electronic Information and Communications, Huazhong University of Science and Technology(华中科技大学电子信息与通信学院);KEENON Robotics Co., Ltd.(科语机器人有限公司);Suzhou Silicon Era Intelligent Technology Co., Ltd.(苏州硅时代智能科技有限公司);College of Materials, Xiamen University(厦门大学材料学院);School of Artificial Intelligence and Automation, Huazhong University of Science and Technology(华中科技大学人工智能与自动化学院);ByteDance(字节跳动);State Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence (BIGAI)(北京通用人工智能研究院通用人工智能国家重点实验室);Hubei Automation Institute(湖北省自动化研究所)
FUNCanon: Learning Pose-Aware Action Primitives via Functional Object Canonicalization for Generalizable Robotic Manipulation
TAMS (Technical Aspects of Multimodal Systems), Department of Informatics, University of Hamburg(汉堡大学信息学院TAMS(多模态系统技术));Technical University of Munich(慕尼黑技术大学);Agile Robots SE(敏捷机器人有限公司)
Decompose and Recompose: Reasoning New Skills from Existing Abilities for Cross-Task Robotic Manipulation
School of Artificial Intelligence, College of Intelligence and Computing, Tianjin University, China(人工智能学院,智能与计算学院,天津大学,中国);School of Computer Science and Information Engineering, Hefei University of Technology, China(计算机科学与信息工程学院,合肥工业大学,中国)
本文提出Decompose and Recompose框架,通过分解现有任务演示为可解释的技能-动作对,实现跨任务机械操作的零样本泛化。
ManipDreamer: Boosting Robotic Manipulation World Model with Action Tree and Visual Guidance
State Key Laboratory of Multimedia Information Processing(多媒体信息处理国家重点实验室);School of Computer Science, Peking University(北京大学计算机学院);Hong Kong University of Science and Technology(香港科技大学);Autonomous Driving Development, NIO(蔚来自动驾驶研发)
Unsupervised Skill Discovery for Robotic Manipulation through Automatic Task Generation
NAVER LABS Europe(NAVER欧洲实验室);TU Darmstadt(达姆施塔特工业大学);EPITA Research Laboratory (LRE)(EPITA研究实验室);German Research Center for AI (DFKI)(德国人工智能研究中心);Center for Cognitive Science, TU Darmstadt(达姆施塔特工业大学认知科学中心);Hessian Center for Artificial Intelligence (Hessian.ai)(黑森州人工智能中心)
DROM: A Language-Guided Diffusion Framework for Multi-Skill Robotic Manipulation
University of Applied Science and Arts of Southern Switzerland (SUPSI)(瑞士南部应用科学与艺术大学);Institute of Systems and Technologies for Sustainable Production (ISTePS)(可持续生产系统与技术研究所);Istituto Dalle Molle di studi sull’intelligenza artificiale (IDSIA)(达勒莫尔人工智能研究所)
Generalizable Long-Horizon Manipulations with Large Language Models
National University of Singapore(新加坡国立大学);UC Berkeley(加州大学伯克利分校);University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校);MIT-IBM Watson AI Lab(麻省理工学院-IBM沃森人工智能实验室)
Language Movement Primitives: Grounding Language Models in Robot Motion
Collab , Dept. of Mechanical Engineering, Virginia Tech, Blacksburg, VA 24061(合作组,机械工程系,弗吉尼亚理工学院,黑斯堡,VA 24061);TEA Lab , Dept. of Mechanical Engineering, Virginia Tech, Blacksburg, VA 24061(TEA实验室,机械工程系,弗吉尼亚理工学院,黑斯堡,VA 24061)
A Generalized Robotic Handwriting Learning System based on Dynamic Movement Primitives (DMPs)
Georgia Institute of Technology(佐治亚理工学院)
示教学习(Learning from demonstration,LfD)是一种功能强大的学习方法,它能够使机器人在一个或多个人类演示的情况下推断出如何执行所需任务。通过从最终用户演示中学习,而不是要求领域专家手动编程每项技能,机器人可以更容易地应用于更广泛的现实世界应用。书写机器人作为LfD的应用之一,由于人类手写轨迹的复杂性,已经成为一个具有挑战性的研究课题。在本文中,我们介绍了一个广义的手写学习系统的物理机器人学习从人类的笔迹的例子,以绘制字母数字字符。我们的机器人系统能够模仿人类演示者的书写方式重写字母,并以类似的书写风格创建新的字母。对于这个系统,我们开发了一个增强的动态运动基元(dynamic movement primitive,简称MOM)算法,MOM *,它增强了我们的机器人系统的鲁棒性和泛化能力。
Reinforcement Learning with Parameterized Manipulation Primitives for Robotic Assembly
Singapore Centre for 3D Printing (SC3DP)(新加坡3D打印中心(SC3DP));School of Mechanical and Aerospace Engineering, NTU(南洋理工大学机械与航空航天工程学院);Eureka Robotics(尤里卡机器人公司)