Simultaneous locomotion and manipulation enables robots to interact with their environment beyond the constraints of a fixed base. However, coordinating legged locomotion with arm manipulation, while considering safety and compliance during contact interaction remains challenging. To this end, we propose a whole-body controller that combines a model-based admittance control for the manipulator arm with a Reinforcement Learning (RL) policy for legged locomotion. The admittance controller maps external wrenches--such as those applied by a human during physical interaction--into desired end-effector velocities, allowing for compliant behavior. The velocities are tracked jointly by the arm and leg controllers, enabling a unified 6-DoF force response. The model-based design permits accurate force control and safety guarantees via a Reference Governor (RG), while robustness is further improved by a Kalman filter enhanced with neural networks for reliable base velocity estimation. We validate our approach in both simulation and hardware using the Unitree Go2 quadruped robot with a 6-DoF arm and wrist-mounted 6-DoF Force/Torque sensor. Results demonstrate accurate tracking of interaction-driven velocities, compliant behavior, and safe, reliable performance in dynamic settings.
This paper presents a novel method to control humanoid robot dynamic loco-manipulation with multiple contact modes via multi-contact Model Predictive Control (MPC) framework. The proposed framework includes a multi-contact dynamics model capable of capturing various contact modes in loco-manipulation, such as hand-object contact and foot-ground contacts. Our proposed dynamics model represents the object dynamics as an external force acting on the system, which simplifies the model and makes it feasible for solving the MPC problem. In numerical validations, our multi-contact MPC framework only …
Recent studies on quadruped robots have focused on either locomotion or mobile manipulation using a robotic arm. Legged robots can manipulate heavier and larger objects using non-prehensile manipulation primitives, such as planar pushing, to drive the object to the desired location. In this paper, we present a novel hierarchical model predictive control (MPC) for contact optimization of the manipulation task. Using two cascading MPCs, we split the loco-manipulation problem into two parts: the first to optimize both contact force and contact location between the robot and the object, and the se…
Efficient Learning of A Unified Policy For Whole-body Manipulation and Locomotion Skills
Institute of Cyber-Systems and Control, Zhejiang University（浙江大学智能系统与控制研究院）；State Key Laboratory of Industrial Control Technology of Zhejiang University（浙江大学工业控制技术状态重点实验室）；Department of Computer Science, University College London（伦敦大学学院计算机科学系）
Deep reinforcement learning produces robust locomotion policies for legged robots over challenging terrains. To date, few studies have leveraged model-based methods to combine these locomotion skills with the precise control of manipulators. Here, we incorporate external dynamics plans into learning-based locomotion policies for mobile manipulation. We train the base policy by applying a random wrench sequence on the robot base in simulation and adding the noisified wrench sequence prediction to the policy observations. The policy then learns to counteract the partially-known future disturbanc…
RoboDuet: Learning a Cooperative Policy for Whole-body Legged Loco-Manipulation
Tsinghua University（清华大学）；University of San Diego（圣地亚哥大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）；Shanghai Qi Zhi Institute（上海期智研究院）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；Jianghuai Advance Technology Center（江淮先进技术中心）
Legged robots have shown remarkable advantages in navigating uneven terrain. However, realizing effective locomotion and manipulation tasks on quadruped robots is still challenging. In addition, object and terrain parameters are generally unknown to the robot in these problems. Therefore, this paper proposes a hierarchical adaptive control framework that enables legged robots to perform loco-manipulation tasks without any given assumption on the object's mass, the friction coefficient, or the slope of the terrain. In our approach, we first present an adaptive manipulation control to regulate t…
X2-N: A Transformable Wheel-legged Humanoid Robot with Dual-mode Locomotion and Manipulation
Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology（香港理工大学电子与计算机工程系）；X-lab Department, Agibot Innovation Company（阿吉博创新公司X实验室）
RAMBO: RL-Augmented Model-Based Whole-Body Control for Loco-Manipulation
Computational Robotics Lab in the Department of Computer Science, ETH Zurich（苏黎世联邦理工学院计算机科学系计算机器人实验室）；Robotics Institute and the School of Computer Science at Carnegie Mellon University（卡内基梅隆大学机器人研究所和计算机科学学院）
ULC: A Unified and Fine-Grained Controller for Humanoid Loco-Manipulation
State Key Laboratory of Robotics and Systems, Harbin Institute of Technology（机器人与系统国家重点实验室，哈尔滨工业大学）；School of Engineering, Westlake University（工程学院，西湖大学）
An attached arm can significantly increase the applicability of legged robots to several mobile manipulation tasks that are not possible for the wheeled or tracked counterparts. The standard hierarchical control pipeline for such legged manipulators is to decouple the controller into that of manipulation and locomotion. However, this is ineffective. It requires immense engineering to support coordination between the arm and legs, and error can propagate across modules causing non-smooth unnatural motions. It is also biological implausible given evidence for strong motor synergies across limbs.…
This study proposes a hierarchically integrated framework for safe task and motion planning (TAMP) of bipedal locomotion in a partially observable environment with dynamic obstacles and uneven terrain. The high-level task planner employs linear temporal logic (LTL) for a reactive game synthesis between the robot and its environment and provides a formal guarantee on navigation safety and task completion. To address environmental partial observability, a belief abstraction is employed at the high-level navigation planner to estimate the dynamic obstacles' location. Accordingly, a synthesized ac…
PILOT: A Perceptive Integrated Low-level Controller for Loco-manipulation over Unstructured Scenes
School of Automation and Intelligent Sensing（自动化与智能感知学院）；Shanghai Jiao Tong University（上海交通大学）；Global College（全球学院）；school of Computer Science（计算机科学学院）；Shanghai Key Laboratory of Navigation and Location Based Services（导航与位置基于服务上海市重点实验室）
PAKE: Learning Whole-Body Loco-Manipulation with Partial Kinematic Embeddings
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Korea Advanced Institute of Science and Technology（韩国科学技术院）；ETH Zurich（苏黎世联邦理工学院）
ViLoMan: Learning Visual-Proprioceptive Whole-Body Loco-Manipulation Skills for Humanoid Robots
State Key Laboratory of AI Safety, Institute of Computing Technology, CAS（中国科学院计算技术研究所人工智能安全国家重点实验室）；University of Chinese Academy of Sciences (CAS)（中国科学院大学）；Beijing Academy of Artificial Intelligence (BAAI)（北京智源人工智能研究院）
In this paper, we present a real-time whole-body planner for collision-free legged mobile manipulation. We enforce both self-collision and environment-collision avoidance as soft constraints within a Model Predictive Control (MPC) scheme that solves a multi-contact optimal control problem. By penalizing the signed distances among a set of representative primitive collision bodies, the robot is able to safely execute a variety of dynamic maneuvers while preventing any self-collisions. Moreover, collision-free navigation and manipulation in both static and dynamic environments are made viable th…
MLM: Learning Multi-task Loco-Manipulation Whole-Body Control for Quadruped Robot with Arm
School of Mechanical Engineering, Shanghai Jiao Tong University（上海交通大学机械工程学院）；Shanghai AI Laboratory（上海人工智能实验室）；Lenovo Corporation（联想公司）；Institute of Artificial Intelligence (TeleAI), China Telecom（中国电信人工智能研究院（TeleAI））
Whole-Body Inverse Dynamics MPC for Legged Loco-Manipulation
Department of Mechanical and Process Engineering, ETH Zurich（机械与过程工程系，苏黎世联邦理工学院）；Computational Robotics Lab, Department of Computer Science, ETH Zurich（计算机器人实验室，计算机科学系，苏黎世联邦理工学院）
A Nonlinear MPC Framework for Loco-Manipulation of Quadrupedal Robots with Non-Negligible Manipulator Dynamics
Department of Mechanical Engineering, Virginia Tech（弗吉尼亚理工大学机械工程系）；California Institute of Technology（加州理工学院）；School of Neuroscience, Virginia Tech（弗吉尼亚理工大学神经科学学院）
Shield-Loco: Shielding Locomotion Policies with Predictive Safety Filtering
Robert Bosch Center for Cyber Physical Systems（罗伯特·博世网络物理系统中心）；Indian Institute of Science（印度科学研究院）；Munich Institute of Robotics and Machine Intelligence (MIRMI)（慕尼黑机器人与机器智能研究所（MIRMI））；Technical University of Munich（慕尼黑技术大学）；Department of Computer Science & Automation（计算机科学与自动化部门）；Department of Mechanical Engineering（机械工程系）；Carnegie Mellon University（卡内基梅隆大学）；Institute for Advanced Study（高级研究 institute）
Kinematics-Aware Multi-Policy Reinforcement Learning for Force-Capable Humanoid Loco-Manipulation
the Robot and Artificial Intellifence Lab, College of Electronic and Information Engineering, Tongji University, Shanghai 201804, China（机器人与人工智能实验室，电子与信息工程学院，同济大学，上海）；State Key Laboratory of Autonomous Intelligent Unmanned System(Tongji University), Shanghai, China（自主智能无人系统国家重点实验室（同济大学），上海）
This study proposes an integrated task and motion planning method for dynamic locomotion in partially observable environments with multi-level safety guarantees. This layered planning framework is composed of a high-level symbolic task planner and a low-level phase-space motion planner. A belief abstraction at the task planning level enables belief estimation of dynamic obstacle locations and guarantees navigation safety with collision avoidance. The high-level task planner, i.e., a two-level navigation planner, employs linear temporal logic for a reactive game synthesis between the robot and …
Enforcing safety in the presence of stochastic uncertainty is a challenging problem. Traditionally, researchers have proposed safety in the statistical mean as a safety measure in this case. However, ensuring safety in the statistical mean is only reasonable if system's safe behavior in the large number of runs is of interest, which precludes the use of mean safety in practical scenarios. In this paper, we propose a risk sensitive notion of safety called conditional-value-at-risk (CVaR) safety, which is concerned with safe performance in the worst case realizations. We introduce CVaR barrier f…
