Reliable terrain perception is a critical prerequisite for the deployment of humanoid robots in unstructured, human-centric environments. While traditional systems often rely on manually engineered, single-sensor pipelines, this paper presents a learning-based framework that uses an intermediate, robot-centric heightmap representation. A hybrid Encoder-Decoder Structure (EDS) is introduced, utilizing a Convolutional Neural Network (CNN) for spatial feature extraction fused with a Gated Recurrent Unit (GRU) core for temporal consistency. The architecture integrates multimodal data from an Intel RealSense depth camera, a LIVOX MID-360 LiDAR processed via efficient spherical projection, and an onboard IMU. Quantitative results demonstrate that multimodal fusion improves reconstruction accuracy by 7.2% over depth-only and 9.9% over LiDAR-only configurations. Furthermore, the integration of a 3.2 s temporal context reduces mapping drift.
ADP: Adversarial Dynamics Priors for Physically Grounded Humanoid Locomotion
Mechatronics, Systems and Control Lab (MSC Lab), Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院机械工程系机电一体化、系统与控制实验室（MSC实验室））；Samsung Electronics, Future Robotics AI Group（三星电子未来机器人人工智能集团）；School of Electrical Engineering, Hanyang University（汉阳大学电气工程学院）；Advanced Robotics Research Center, Korea Institute of Machinery & Materials (KIMM)（韩国机械与材料研究所先进机器人研究中心）
X2-N: A Transformable Wheel-legged Humanoid Robot with Dual-mode Locomotion and Manipulation
Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology（香港理工大学电子与计算机工程系）；X-lab Department, Agibot Innovation Company（阿吉博创新公司X实验室）
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Carnegie Mellon University（卡内基梅隆大学）
Bilateral teleoperation provides humanoid robots with human planning intelligence while enabling the human to feel what the robot feels. It has the potential to transform physically capable humanoid robots into dynamically intelligent ones. However, dynamic bilateral locomotion teleoperation remains as a challenge due to the complex dynamics it involves. This work presents our initial step to tackle this challenge via the concept of wheeled humanoid robot locomotion teleoperation by body tilt. Specifically, we developed a force-feedback-capable whole-body human-machine interface (HMI), and des…
University of Texas at Austin（德克萨斯大学奥斯汀分校）；Stanford University（斯坦福大学）
Whole-body control (WBC) is a generic task-oriented control method for feedback control of loco-manipulation behaviors in humanoid robots. The combination of WBC and model-based walking controllers has been widely utilized in various humanoid robots. However, to date, the WBC method has not been employed for unsupported passive-ankle dynamic locomotion. As such, in this paper, we devise a new WBC, dubbed whole-body locomotion controller (WBLC), that can achieve experimental dynamic walking on unsupported passive-ankle biped robots. A key aspect of WBLC is the relaxation of contact constraints …
MASH: Cooperative-Heterogeneous Multi-Agent Reinforcement Learning for Single Humanoid Robot Locomotion
Faculty of Robot Science and Engineering, Northeastern University（机器人科学与工程学院，东北大学）；Guangdong Key Laboratory of Intelligent Morphing Mechanisms and Adaptive Robotics and School of Intelligence Science and Engineering, the Harbin Institute of Technology Shenzhen（智能变形机制与适应机器人重点实验室和智能科学与工程学院，哈尔滨工业大学深圳）
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；K. N. Toosi University of Technology（K.N.托西理工大学）；Tandon School of Engineering, New York University（纽约大学坦登工程学院）
Trajectory optimization (TO) is one of the most powerful tools for generating feasible motions for humanoid robots. However, including uncertainties and stochasticity in the TO problem to generate robust motions can easily lead to an interactable problem. Furthermore, since the models used in the TO have always some level of abstraction, it is hard to find a realistic set of uncertainty in the space of abstract model. In this paper we aim at leveraging a sample-efficient learning technique (Bayesian optimization) to robustify trajectory optimization for humanoid locomotion. The main idea is to…
Istituto Italiano di Tecnologia（意大利理工学院）；University of Genoa（热那亚大学）；University of Manchester（曼彻斯特大学）
This paper presents a Non-Linear Model Predictive Controller for humanoid robot locomotion with online step adjustment capabilities. The proposed controller considers the Centroidal Dynamics of the system to compute the desired contact forces and torques and contact locations. Differently from bipedal walking architectures based on simplified models, the presented approach considers the reduced centroidal model, thus allowing the robot to perform highly dynamic movements while keeping the control problem still treatable online. We show that the proposed controller can automatically adjust the …
HWC-Loco: A Hierarchical Whole-Body Control Approach to Robust Humanoid Locomotion
The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；Harbin Institute of Technology（哈尔滨工业大学）；DexForce Technology（DexForce科技）；Southeast University（东南大学）
Fondazione Istituto Italiano di Tecnologia（意大利技术研究院基金会）；Università di Genova（热那亚大学）
Humanoid robots are machines built with an anthropomorphic shape. Despite decades of research into the subject, it is still challenging to tackle the robot locomotion problem from an algorithmic point of view. For example, these machines cannot achieve a constant forward body movement without exploiting contacts with the environment. The reactive forces resulting from the contacts are subject to strong limitations, complicating the design of control laws. As a consequence, the generation of humanoid motions requires to exploit fully the mathematical model of the robot in contact with the envir…
PolygMap: A Perceptive Locomotion Framework for Humanoid Robot Stair Climbing
The Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳人工智能与机器人社会研究院）；Electronic science and technology, Guangdong University of Technology（广东技术大学电子科学与技术学院）；Leju robotics（乐居机器人）
This work presents a teleoperated humanoid robot system that can imitate human motions, walk and turn. To capture human motions, a Microsoft Kinect Depth sensor is used. Unlike the cumbersome motion capture suits, the sensor makes the system more comfortable to interact with. The skeleton data is extracted from the Kinect which is processed to generate the robot's joint angles. Robot Operating System (ROS) is used for the communication between the various parts of the code to achieve minimal latency. Thus the robot imitates the human in real-time with negligible lag. Unlike most of the human m…
Cost-Matching Model Predictive Control for Efficient Reinforcement Learning in Humanoid Locomotion
New York University Abu Dhabi (NYUAD)（纽约大学阿布扎比分校）；Georgia Institute of Technology（佐治亚理工学院）；Norwegian University of Science and Technology (NTNU)（挪威科技大学）
StyleLoco: Generative Adversarial Distillation for Natural Humanoid Robot Locomotion
National Key Laboratory of General Artificial Intelligence, BIGAI（通用人工智能国家重点实验室，BIGAI）；School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）；Huazhong University of Science and Technology（华中科技大学）
VB-Com: Learning Vision-Blind Composite Humanoid Locomotion Against Deficient Perception
Shanghai AI Laboratory（上海人工智能实验室）；The University of Hong Kong（香港大学）；Shanghai Jiao Tong University（上海交通大学）；Zhejiang University（浙江大学）；The Chinese University of Hong Kong（香港中文大学）
DoublyAware: Dual Planning and Policy Awareness for Temporal Difference Learning in Humanoid Locomotion
University of Texas at Arlington（德克萨斯大学阿灵顿分校）；Intelligent Autonomous Systems Lab（智能自主系统实验室）；German Research Center for AI (DFKI)（德国人工智能研究中心）；Automation & Control Institute (ACIN)（自动化与控制研究所）；Austrian Institute of Technology (AIT) GmbH（奥地利技术研究所）
University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校）；Purdue University（普渡大学）；University of Rhode Island（罗德岛大学）
State estimation for legged locomotion over a dynamic rigid surface (DRS), which is a rigid surface moving in the world frame (e.g., ships, aircraft, and trains), remains an under-explored problem. This paper introduces an invariant extended Kalman filter that estimates the robot's pose and velocity during DRS locomotion by using common sensors of legged robots (e.g., inertial measurement units (IMU), joint encoders, and RDB-D camera). A key feature of the filter lies in that it explicitly addresses the nonstationary surface-foot contact point and the hybrid robot behaviors. Another key featur…
Italian Institute of Technology（意大利技术研究院）；University of Manchester（曼彻斯特大学）；University of Genoa（热那亚大学）
The paper presents a planner to generate walking trajectories by using the centroidal dynamics and the full kinematics of a humanoid robot. The interaction between the robot and the walking surface is modeled explicitly via new conditions, the \emph{Dynamical Complementarity Constraints}. The approach does not require a predefined contact sequence and generates the footsteps automatically. We characterize the robot control objective via a set of tasks, and we address it by solving an optimal control problem. We show that it is possible to achieve walking motions automatically by specifying a m…
Institute for Infocomm Research, A*STAR（新加坡科技研究局资讯通信研究院）；University College London（伦敦大学学院）
This work developed a learning framework for perceptive legged locomotion that combines visual feedback, proprioceptive information, and active gait regulation of foot-ground contacts. The perception requires only one forward-facing camera to obtain the heightmap, and the active regulation of gait paces and traveling velocity are realized through our formulation of CPG-based high-level imitation of foot-ground contacts. Through this framework, an end-user has the ability to command task-level inputs to control different walking speeds and gait frequencies according to the traversal of differen…
Adversarial Locomotion and Motion Imitation for Humanoid Policy Learning
Institute of Artificial Intelligence (TeleAI), China Telecom（人工智能研究院（TeleAI），中国电信）；ShanghaiTech University（上海科技大学）；University of Science and Technology of China（中国科学技术大学）；Northwestern Polytechnical University（西北工业大学）；Tsinghua University（清华大学）
K. N. Toosi University of Technology（K. N. 图西理工大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；University of Trento（特伦托大学）；New York University（纽约大学）
Trajectory optimization (TO) is one of the most powerful tools for generating feasible motions for humanoid robots. However, including uncertainties and stochasticity in the TO problem to generate robust motions can easily lead to intractable problems. Furthermore, since the models used in TO have always some level of abstraction, it can be hard to find a realistic set of uncertainties in the model space. In this paper we leverage a sample-efficient learning technique (Bayesian optimization) to robustify TO for humanoid locomotion. The main idea is to use data from full-body simulations to mak…
TD-GRPC: Temporal Difference Learning with Group Relative Policy Constraint for Humanoid Locomotion
University of Texas at Arlington（德克萨斯大学阿灵顿分校）；VinRobotics；Intelligent Autonomous Systems Lab, TU Darmstadt（图宾根大学智能自主系统实验室）；German Research Center for AI (DFKI)（德国人工智能研究中心）；Automation & Control Institute (ACIN), TU Wien（维也纳技术大学自动化与控制研究所）
Humanoid Locomotion and Manipulation: Current Progress and Challenges in Control, Planning, and Learning
Georgia Institute of Technology（佐治亚理工学院）；The University of Southern California（南加州大学）；Technische Universität München（慕尼黑技术大学）；Google DeepMind（谷歌DeepMind）；The AI Institute（人工智能研究所）；The Institute for Human and Machine Cognition（人机认知研究所）；Duke University（杜克大学）；Kuwait University（科威特大学）；Stanford University（斯坦福大学）；CNRS-University of Montpellier（国家科学研究中心-蒙彼利埃大学）；LIRMM CNRS-AIST Joint Robotics Laboratory（联合机器人实验室）；Simon Fraser University（西蒙弗雷泽大学）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；NVIDIA（英伟达）；Carnegie Mellon University（卡内基梅隆大学）；Harbin Institute of Technology（哈尔滨工业大学）
Gait-Adaptive Perceptive Humanoid Locomotion with Real-Time Under-Base Terrain Reconstruction
Department of Electronic Engineering and Information Science (EEIS), University of Science and Technology of China（电子工程与信息科学系，中国科学技术大学）；LimX Dynamics（LimX动力学）；Hong Kong University of Science and Technology（香港科技大学）；School of Mechanics Engineering, Harbin Institute of Technology (HIT)（机械工程学院，哈尔滨工业大学）；Department of Computing, The Hong Kong Polytechnic University（计算学院，香港理工大学）；Zhejiang University-University of Illinois Urbana-Champaign Institute (ZJUI)（浙江大学-伊利诺伊大学厄巴纳-香槟分校联合研究所）
PGTT: Phase-Guided Terrain Traversal for Perceptive Legged Locomotion
Laboratory of Automation and Robotics (LAR) in the Department of Electrical & Computer Engineering, University of Patras（自动化与机器人实验室（LAR）（电气与计算机工程系，帕特拉斯大学））；Archimedes/Athena RC, Greece（阿基米德/雅典娜RC，希腊）；Athena - Research and Innovation Center in Information, Communication and Knowledge Technologies, Xanthi, Greece（雅典娜信息、通信和知识技术研究中心（Xanthi，希腊））；Computational Intelligence Laboratory (CILab), Department of Mathematics, University of Patras（计算智能实验室（CILab）（数学系，帕特拉斯大学））
Locomotion for legged robots poses considerable challenges when confronted by obstacles and adverse environments. Footstep planners are typically only designed for one mode of locomotion, but traversing unfavorable environments may require several forms of locomotion to be sequenced together, such as walking, crawling, and jumping. Multi-modal motion planners can be used to address some of these problems, but existing implementations tend to be time-consuming and are limited to quasi-static actions. This paper presents a motion planning method to traverse complex environments using multiple ca…
IEETA / DETI University of Aveiro（阿威罗大学 IEETA / DETI）
Generating robust locomotion for a humanoid robot in the presence of disturbances is difficult because of its high number of degrees of freedom and its unstable nature. In this paper, we used the concept of Divergent Component of Motion~(DCM) and propose an optimal closed-loop controller based on Linear-Quadratic-Gaussian to generate a robust and stable walking for humanoid robots. The biped robot dynamics has been approximated using the Linear Inverted Pendulum Model~(LIPM). Moreover, we propose a controller to adjust the landing location of the swing leg to increase the withstanding level of…
Humanoid robots are increasingly demanded to operate in interactive and human-surrounded environments while achieving sophisticated locomotion and manipulation tasks. To accomplish these tasks, roboticists unremittingly seek for advanced methods that generate whole-body coordination behaviors and meanwhile fulfill various planning and control objectives. Undoubtedly, these goals pose fundamental challenges to the robotics and control community. To take an incremental step towards reducing the performance gap between theoretical foundations and real implementations, we present a planning and co…
