Synthesizing complex whole-body manipulation behaviors has fundamental challenges due to the rapidly growing combinatorics inherent to contact interaction planning. While model-based methods have shown promising results in solving long-horizon manipulation tasks, they often work under strict assumptions, such as known model parameters, oracular observation of the environment state, and simplified dynamics, resulting in plans that cannot easily transfer to hardware. Learning-based approaches, such as imitation learning (IL) and reinforcement learning (RL), have been shown to be robust when operating over in-distribution states; however, they need heavy human supervision. Specifically, model-free RL requires a tedious reward-shaping process. IL methods, on the other hand, rely on human demonstrations that involve advanced teleoperation methods. In this work, we propose a plan-guided reinforcement learning (PGRL) framework to combine the advantages of model-based planning and reinforcement learning. Our method requires minimal human supervision because it relies on plans generated by model-based planners to guide the exploration in RL. In exchange, RL derives a more robust policy thanks to domain randomization. We test this approach on a whole-body manipulation task on Punyo, an upper-body humanoid robot with compliant, air-filled arm coverings, to pivot and lift a large box. Our preliminary results indicate that the proposed methodology is promising to address challenges that remain difficult for either model- or learning-based strategies alone.
TACT: Humanoid Whole-body Contact Manipulation through Deep Imitation Learning with Tactile Modality
CNRS-AIST JRL (Joint Robotics Laboratory), IRL and National Institute of Advanced Industrial Science and Technology (AIST)(CNRS-AIST联合机器人实验室、IRL和国家先进工业科学与技术研究院);Tokyo University of Science(东京科学大学);University of Tsukuba(茨口大学)
Integrating Learning-Based Manipulation and Physics-Based Locomotion for Whole-Body Badminton Robot Control
School of Computer Science and Technology, Shandong University(山东大学计算机科学与技术学院);Robotics Institute, Carnegie Mellon University(卡内基梅隆大学机器人研究所);DeepCode Robotics(DeepCode机器人公司);Department of Sports, Shanghai Jiao Tong University(上海交通大学体育学院)
RoboDuet: Learning a Cooperative Policy for Whole-body Legged Loco-Manipulation
Tsinghua University(清华大学);University of San Diego(圣地亚哥大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Institute for Interdisciplinary Information Sciences, Tsinghua University(清华大学交叉信息研究院);Shanghai Qi Zhi Institute(上海期智研究院);Tsinghua Shenzhen International Graduate School(清华大学深圳国际研究生院);Jianghuai Advance Technology Center(江淮先进技术中心)
CWI: Composite Humanoid Whole-Body Imitation System for Loco-manipulation
LimX Dynamics;The University of Hong Kong(香港大学);School of Automation and Intelligent Manufacturing, Southern University of Science and Technology(南方科技大学自动化与智能制造学院);Hong Kong University of Science and Technology(香港科技大学);ZJU-UIUC Institute, Zhejiang University(浙江大学伊利诺伊大学厄巴纳香槟校区联合学院)
DexRoam: Learning Mobile Bimanual Dexterous Manipulation from Egocentric Whole-Body Human Demonstrations
The Hong Kong University of Science and Technology(香港科技大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院);State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(北京大学计算机学院多媒体信息处理国家重点实验室);Beihang University(北京航空航天大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
Whole-Body Inverse Dynamics MPC for Legged Loco-Manipulation
Department of Mechanical and Process Engineering, ETH Zurich(机械与过程工程系,苏黎世联邦理工学院);Computational Robotics Lab, Department of Computer Science, ETH Zurich(计算机器人实验室,计算机科学系,苏黎世联邦理工学院)
ViLoMan: Learning Visual-Proprioceptive Whole-Body Loco-Manipulation Skills for Humanoid Robots
State Key Laboratory of AI Safety, Institute of Computing Technology, CAS(中国科学院计算技术研究所人工智能安全国家重点实验室);University of Chinese Academy of Sciences (CAS)(中国科学院大学);Beijing Academy of Artificial Intelligence (BAAI)(北京智源人工智能研究院)
MLM: Learning Multi-task Loco-Manipulation Whole-Body Control for Quadruped Robot with Arm
School of Mechanical Engineering, Shanghai Jiao Tong University(上海交通大学机械工程学院);Shanghai AI Laboratory(上海人工智能实验室);Lenovo Corporation(联想公司);Institute of Artificial Intelligence (TeleAI), China Telecom(中国电信人工智能研究院(TeleAI))
Zero-shot Whole-Body Manipulation with a Large-Scale Soft Robotic Torso via Guided Reinforcement Learning
Department of Mechanical Engineering, Brigham Young University(机械工程系, Brigham Young 大学);Humanoid Sensing and Perception, Italian Institute of Technology(人形感知与感知,意大利理工研究院);The BioRobotics Institute, Scuola Superiore Sant’Anna(生物机器人研究所,圣安娜高等学院)
本文构建高速 MuJoCo 软体机器人仿真,并用运动原语引导强化学习,首次在 Baloo 平台实现双臂六自由度、10 kg 负载全身操作的零样本 sim-to-real 迁移,成功率达 88%。
RAMBO: RL-Augmented Model-Based Whole-Body Control for Loco-Manipulation
Computational Robotics Lab in the Department of Computer Science, ETH Zurich(苏黎世联邦理工学院计算机科学系计算机器人实验室);Robotics Institute and the School of Computer Science at Carnegie Mellon University(卡内基梅隆大学机器人研究所和计算机科学学院)
PAKE: Learning Whole-Body Loco-Manipulation with Partial Kinematic Embeddings
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Korea Advanced Institute of Science and Technology(韩国科学技术院);ETH Zurich(苏黎世联邦理工学院)
Towards a General Humanoid Loco-Manipulation Model via Egocentric Whole-Body Human Data Pretraining
Alibaba Group(阿里巴巴集团);Sichuan University(四川大学);Shanghai Innovation Institute(上海创新研究院);Beihang University(北京航空航天大学);Hong Kong University of Science and Technology(香港科技大学)
HeLoM: Hierarchical Learning for Whole-Body Loco-Manipulation by a Hexapod Robot
MARMoT Lab, Department of Mechanical Engineering, National University of Singapore(机械工程系,新加坡国立大学MARMoT实验室);Center for X-mechanics, Zhejiang University(浙江大学X力学中心)
Blind Dexterity: Whole-Body Humanoid Manipulation via Pure Proprioception
TU Darmstadt(达姆施塔特工业大学);German Research Center for AI (DFKI)(德国人工智能研究中心);Tongji University(同济大学);Shanghai Research Institute for Autonomous Intelligent Systems(上海自主智能系统研究院);Robotics Institute Germany(德国机器人研究所)