Human Demonstrations are Generalizable Knowledge for Robots
School of Automation, Beijing Institute of Technology, Beijing, 100081, China（自动化学院，北京理工大学，北京，100081，中国）；Beijing Zhongguancun Academy, Beijing, 100094, China（中关村学院，北京，100094，中国）
Learning from human demonstrations is an emerging trend for designing intelligent robotic systems. However, previous methods typically regard videos as instructions, simply dividing them into action sequences for robotic repetition, which poses obstacles to generalization to diverse tasks or object instances. In this paper, we propose a different perspective, considering human demonstration videos not as mere instructions, but as a source of knowledge for robots. Motivated by this perspective and the remarkable comprehension and generalization capabilities exhibited by large language models (LLMs), we propose DigKnow, a method that DIstills Generalizable KNOWledge with a hierarchical structure. Specifically, DigKnow begins by converting human demonstration video frames into observation knowledge. This knowledge is then subjected to analysis to extract human action knowledge and further distilled into pattern knowledge compassing task and object instances, resulting in the acquisition of generalizable knowledge with a hierarchical structure. In settings with different tasks or object instances, DigKnow retrieves relevant knowledge for the current task and object instances. Subsequently, the LLM-based planner conducts planning based on the retrieved knowledge, and the policy executes actions in line with the plan to achieve the designated task. Utilizing the retrieved knowledge, we validate and rectify planning and execution outcomes, resulting in a substantial enhancement of the success rate. Experimental results across a range of tasks and scenes demonstrate the effectiveness of this approach in facilitating real-world robots to accomplish tasks with the knowledge derived from human demonstrations.
Learning Generalizable Language-Conditioned Cloth Manipulation from Long Demonstrations
Center for Artificial Intelligence and Robotics, Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院人工智能与机器人中心）；School of Computing, National University of Singapore（新加坡国立大学计算机学院）
Generalizable Skill Learning for Construction Robots with Crowdsourced Natural Language Instructions, Composable Skills Standardization, and Large Language Model
Virginia Tech（弗吉尼亚理工大学）；University of Michigan（密歇根大学）
Reinforcement learning in linear embedding space unlocks generalizable control across soft robot configurations
National University of Defense Technology（国防科技大学）；Hefei University of Technology（合肥工业大学）；Nanjing University (Suzhou Campus)（南京大学（苏州校区））；Technical University of Munich（慕尼黑工业大学）；Beihang University（北京航空航天大学）；Newcastle University（纽卡斯尔大学）
Learning Generalizable 3D Manipulation With 10 Demonstrations
State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所 机器人学国家重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；College of Automation Science and Engineering, South China University of Technology（华南理工大学自动化科学与工程学院）
RwoR: Generating Robot Demonstrations from Human Hand Collection for Policy Learning without Robot
CFCS, School of Computer Science, Peking University（计算机学院，北京大学）；PKU-Agibot Lab（北京大学Agibot实验室）；State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家重点实验室，计算机学院，北京大学）；Tencent Robotics X Laboratory（腾讯机器人X实验室）
LLM-Based Generalizable Hierarchical Task Planning and Execution for Heterogeneous Robot Teams with Event-Driven Replanning
Department of Mechanical Engineering, IIT Gandhinagar（印度加尔各直辖区理工学院机械工程系）；Sahyadri College of Engineering and Management（萨哈亚德里工程与管理学院）；Vishwakarma Institute of Information Technology（维什瓦克arma信息技术学院）
Inferring the Geometric Nullspace of Robot Skills from Human Demonstrations
A*STAR Institute for Infocomm Research (I2R)（新加坡科技研究局资讯通信研究院）；A*STAR Human-centric AI (CHEEM) Programme, Institute of High Performance Computing（新加坡科技研究局高性能计算研究院人类中心人工智能项目）；A*STAR Advanced Remanufacturing and Technology Centre (ARTC)（新加坡科技研究局先进再制造与技术中心）
DFL-TORO: A One-Shot Demonstration Framework for Learning Time-Optimal Robotic Manufacturing Tasks
University of Luxembourg（卢森堡大学）；Interdisciplinary Centre for Security, Reliability, and Trust (SnT)（安全、可靠性与跨学科信任中心（SnT））；Faculty of Science, Technology, and Medicine（科学、技术与医学学院）
