Perceiving the surrounding environment is crucial for autonomous mobile robots. An elevation map provides a memory-efficient and simple yet powerful geometric representation for ground robots. The robots can use this information for navigation in an unknown environment or perceptive locomotion control over rough terrain. Depending on the application, various post processing steps may be incorporated, such as smoothing, inpainting or plane segmentation. In this work, we present an elevation mapping pipeline leveraging GPU for fast and efficient processing with additional features both for navigation and locomotion. We demonstrated our mapping framework through extensive hardware experiments. Our mapping software was successfully deployed for underground exploration during DARPA Subterranean Challenge and for various experiments of quadrupedal locomotion.
Norwegian University of Life Sciences（挪威生命科学大学）；University of Lincoln（林肯大学）
This paper introduces a new method for robot motion planning and navigation in uneven environments through a surfel representation of underlying point clouds. The proposed method addresses the shortcomings of state-of-the-art navigation methods by incorporating both kinematic and physical constraints of a robot with standard motion planning algorithms (e.g., those from the Open Motion Planning Library), thus enabling efficient sampling-based planners for challenging uneven terrain navigation on raw point cloud maps. Unlike techniques based on Digital Elevation Maps (DEMs), our novel surfel-bas…
Real-time Neural Dense Elevation Mapping for Urban Terrain with Uncertainty Estimations
Hong Kong University of Science and Technology（香港科技大学）；Clear Water Bay Institute of Autonomous Driving（清水湾自动驾驶研究院）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute（香港科技大学深圳研究院）
Words to Wheels: Vision-Based Autonomous Driving Understanding Human Language Instructions Using Foundation Models
Korea Advanced Institute of Science and Technologies (KAIST)（韩国科学技术院（KAIST））；Electronics and Telecommunications Research Institute (ETRI)（电子通信研究院（ETRI））
WAVE-Go: World-Model Navigation with Adaptive Execution for Wheel-Legged Robots
Beijing Institute of Technology（北京理工大学）；The University of Hong Kong（香港大学）；GAC R&D Center（广汽研究院）；LimX Dynamics；Beijing Academy of Artificial Intelligence (BAAI)（北京人工智能研究院）
Watch Your Step: Learning Semantically-Guided Locomotion in Cluttered Environment
School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）；School of Mechanical and Mining Engineering, The University of Queensland（昆士兰大学机械与采矿工程学院）
Istituto Italiano di Tecnologia（意大利理工学院）；University of Oxford（牛津大学）
Legged robots can outperform wheeled machines for most navigation tasks across unknown and rough terrains. For such tasks, visual feedback is a fundamental asset to provide robots with terrain-awareness. However, robust dynamic locomotion on difficult terrains with real-time performance guarantees remains a challenge. We present here a real-time, dynamic foothold adaptation strategy based on visual feedback. Our method adjusts the landing position of the feet in a fully reactive manner, using only on-board computers and sensors. The correction is computed and executed continuously along the sw…
Shenzhen Institute of Artificial Intelligence and Robotics for Society (AIRS)（深圳市人工智能与机器人研究院）；Institute of Robotics and Intelligent Manufacturing (IRIM), The Chinese University of Hong Kong (CUHK)（香港中文大学机器人与智能制造研究院）；George W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology（佐治亚理工学院乔治·W·伍德拉夫机械工程学院）；Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）
Dynamic quadrupedal locomotion over rough terrains reveals remarkable progress over the last few decades. Small-scale quadruped robots are adequately flexible and adaptable to traverse uneven terrains along sagittal direction, such as slopes and stairs. To accomplish autonomous locomotion navigation in complex environments, spinning is a fundamental yet indispensable functionality for legged robots. However, spinning behaviors of quadruped robots on uneven terrain often exhibit position drifts. Motivated by this problem, this study presents an algorithmic method to enable accurate spinning mot…
REASAN: Learning Reactive Safe Navigation for Legged Robots
Bernoulli Institute for Mathematics, Computer Science and Artificial Intelligence, University of Groningen, The Netherlands（格罗宁根大学伯努利数学、计算机科学与人工智能研究所）；Department of Electrical Engineering, Linköping University, Sweden（利厄普大学电气工程系）；Engineering and Technology Institute Groningen, University of Groningen, The Netherlands（格罗宁根大学格罗宁根工程与技术研究所）
Poznan University of Technology（波兹南理工大学）；University of Edinburgh（爱丁堡大学）
In this paper, we propose a method for selecting the optimal footholds for legged systems. The goal of the proposed method is to find the best foothold for the swing leg on a local elevation map. We apply the Convolutional Neural Network to learn the relationship between the local elevation map and the quality of potential footholds. The proposed network evaluates the geometrical characteristics of each cell on the elevation map, checks kinematic constraints and collisions. During execution time, the controller obtains the qualitative measurement of each potential foothold from the neural mode…
Omnidirectional Humanoid Locomotion on Stairs via Unsafe Stepping Penalty and Sparse LiDAR Elevation Mapping
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）；State Key Laboratory of Intelligent Manufacturing Equipment and Technology, Huazhong University of Science and Technology（华中科技大学智能制造装备技术国家重点实验室）；School of Mechanical Science and Engineering, Huazhong University of Science and Technology（华中科技大学机械科学与工程学院）
We exploit the complementary strengths of vision and proprioception to develop a point-goal navigation system for legged robots, called VP-Nav. Legged systems are capable of traversing more complex terrain than wheeled robots, but to fully utilize this capability, we need a high-level path planner in the navigation system to be aware of the walking capabilities of the low-level locomotion policy in varying environments. We achieve this by using proprioceptive feedback to ensure the safety of the planned path by sensing unexpected obstacles like glass walls, terrain properties like slipperiness…
Building intelligent autonomous systems at any scale is challenging. The sensing and computation constraints of a microrobot platform make the problems harder. We present improvements to learning-based methods for on-board learning of locomotion, classification, and navigation of microrobots. We show how simulated locomotion can be achieved with model-based reinforcement learning via on-board sensor data distilled into control. Next, we introduce a sparse, linear detector and a Dynamic Thresholding method to FAST Visual Odometry for improved navigation in the noisy regime of mm scale imagery. …
Generating natural and physically feasible motions for legged robots has been a challenging problem due to its complex dynamics. In this work, we introduce a novel learning-based framework of autoregressive motion planner (ARMP) for quadruped locomotion and navigation. Our method can generate motion plans with an arbitrary length in an autoregressive fashion, unlike most offline trajectory optimization algorithms for a fixed trajectory length. To this end, we first construct the motion library by solving a dense set of trajectory optimization problems for diverse scenarios and parameter settin…
This study proposes an integrated task and motion planning method for dynamic locomotion in partially observable environments with multi-level safety guarantees. This layered planning framework is composed of a high-level symbolic task planner and a low-level phase-space motion planner. A belief abstraction at the task planning level enables belief estimation of dynamic obstacle locations and guarantees navigation safety with collision avoidance. The high-level task planner, i.e., a two-level navigation planner, employs linear temporal logic for a reactive game synthesis between the robot and …
SoloParkour: Constrained Reinforcement Learning for Visual Locomotion from Privileged Experience
LAAS-CNRS（法国国家科学研究中心自动化与系统分析实验室）；Université de Toulouse（图卢兹大学）；CNRS（法国国家科学研究中心）；New York University（纽约大学）；Artificial and Natural Intelligence Toulouse Institute（图卢兹人工与自然智能研究所）
We present a novel method for reliable robot navigation in uneven outdoor terrains. Our approach employs a novel fully-trained Deep Reinforcement Learning (DRL) network that uses elevation maps of the environment, robot pose, and goal as inputs to compute an attention mask of the environment. The attention mask is used to identify reduced stability regions in the elevation map and is computed using channel and spatial attention modules and a novel reward function. We continuously compute and update a navigation cost-map that encodes the elevation information or the level-of-flatness of the ter…
Gallant: Voxel Grid-based Humanoid Locomotion and Local-navigation across 3D Constrained Terrains
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；The Chinese University of Hong Kong（香港中文大学）；University of Science and Technology of China（中国科学技术大学）；University of Tokyo（东京大学）；Shanghai Jiaotong University（上海交通大学）
Indian Institute of Technology Kharagpur（印度理工学院卡拉格普尔分校）
On the lines of the huge and varied efforts in the field of automation with respect to technology development and innovation of vehicles to make them run autonomously, this paper presents an innovation to a bicycle. A normal daily use bicycle was modified at low cost such that it runs autonomously, while maintaining its original form i.e. the manual drive. Hence, a bicycle which could be normally driven by any human and with a press of switch could run autonomously according to the needs of the user has been developed.
Institute of Robotics and Intelligent Manufacturing (IRIM), The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）机器人与智能制造研究院（IRIM））；Shenzhen Institute of Artificial Intelligence and Robotics for Society (AIRS), The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）深圳人工智能与机器人研究院（AIRS））
Quadruped locomotion is currently a vibrant research area, which has reached a level of maturity and performance that enables some of the most advanced real-world applications with autonomous quadruped robots both in academia and industry. Blind robust quadruped locomotion has been pushed forward in control and technology aspects within recent decades. However, in the complicated environment, the capability including terrain perception and path planning is still required. Visual perception is an indispensable ability in legged locomotion for such a demand. This study explores a vision-based na…
Learning Robust Autonomous Navigation and Locomotion for Wheeled-Legged Robots
Robotic Systems Lab, ETH Zurich（苏黎世联邦理工学院机器人系统实验室）；Neuromeka；Swiss-Mile Robotics AG（Swiss-Mile机器人公司）；The work was carried out during their stay at（该工作是在他们停留期间完成的）
TRAIN: Terrain-Aware Interactive Navigation for Quadruped Robots via Reinforcement Learning
Department of Robotics, Graduate School of Engineering, Tohoku University（东邦大学机器人学系）；Laboratory for Intelligent Decision and Autonomous Robots, Woodruff School of Mechanical Engineering, Georgia Institute of Technology（佐治亚理工学院智能决策与自主机器人实验室）
The common approach for local navigation on challenging environments with legged robots requires path planning, path following and locomotion, which usually requires a locomotion control policy that accurately tracks a commanded velocity. However, by breaking down the navigation problem into these sub-tasks, we limit the robot's capabilities since the individual tasks do not consider the full solution space. In this work, we propose to solve the complete problem by training an end-to-end policy with deep reinforcement learning. Instead of continuously tracking a precomputed path, the robot nee…
We present a novel outdoor navigation algorithm to generate stable and efficient actions to navigate a robot to reach a goal. We use a multi-stage training pipeline and show that our approach produces policies that result in stable and reliable robot navigation on complex terrains. Based on the Proximal Policy Optimization (PPO) algorithm, we developed a novel method to achieve multiple capabilities for outdoor navigation tasks, namely alleviating the robot's drifting, keeping the robot stable on bumpy terrains, avoiding climbing on hills with steep elevation changes, and avoiding collisions. …
Despite the progress in legged robotic locomotion, autonomous navigation in unknown environments remains an open problem. Ideally, the navigation system utilizes the full potential of the robots' locomotion capabilities while operating within safety limits under uncertainty. The robot must sense and analyze the traversability of the surrounding terrain, which depends on the hardware, locomotion control, and terrain properties. It may contain information about the risk, energy, or time consumption needed to traverse the terrain. To avoid hand-crafted traversability cost functions we propose to …
Due to the highly complex environment present during the DARPA Subterranean Challenge, all six funded teams relied on legged robots as part of their robotic team. Their unique locomotion skills of being able to step over obstacles require special considerations for navigation planning. In this work, we present and examine ArtPlanner, the navigation planner used by team CERBERUS during the Finals. It is based on a sampling-based method that determines valid poses with a reachability abstraction and uses learned foothold scores to restrict areas considered safe for stepping. The resulting planni…
Hierarchical Experience-informed Navigation for Multi-modal Quadrupedal Rebar Grid Traversal
Georgia Institute of Technology（佐治亚理工学院）；School of Mechanical Engineering, Georgia Institute of Technology（佐治亚理工学院机械工程学院）；SkyMul Inc（SkyMul公司）；School of Electrical and Computer Engineering, Georgia Institute of Technology（佐治亚理工学院电气与计算机工程学院）
End-to-End Humanoid Robot Safe and Comfortable Locomotion Policy
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Human-Robot Interfaces and Interaction Lab., Istituto Italiano di Tecnologia, Italy（人机交互实验室，意大利理工学院）
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Vanderbilt University（范德堡大学）
We tackle the problem of perceptive locomotion in dynamic environments. In this problem, a quadrupedal robot must exhibit robust and agile walking behaviors in response to environmental clutter and moving obstacles. We present a hierarchical learning framework, named PRELUDE, which decomposes the problem of perceptive locomotion into high-level decision-making to predict navigation commands and low-level gait generation to realize the target commands. In this framework, we train the high-level navigation controller with imitation learning on human demonstrations collected on a steerable cart a…
Zhejiang University（浙江大学）；Huzhou Institute, Zhejiang University（浙江大学湖州研究院）；Harbin Institute of Technology（哈尔滨工业大学）
Terrestrial-aerial bimodal vehicles bloom in both academia and industry because they incorporate both the high mobility of aerial vehicles and the long endurance of ground vehicles. In this work, we present an autonomous and adaptive navigation framework to bring complete autonomy to this class of vehicles. The framework mainly includes 1) a hierarchical motion planner that generates safe and low-power terrestrial-aerial trajectories in unknown environments and 2) a unified motion controller which dynamically adjusts energy consumption in terrestrial locomotion. Extensive real-world experiment…
Identifying Terrain Physical Parameters from Vision -- Towards Physical-Parameter-Aware Locomotion and Navigation
ETH Zurich（苏黎世联邦理工学院）；Max Planck Institute for Intelligent Systems Tübingen（马克斯·普朗克智能系统研究所（图宾根））；Harbin Institute of Technology（哈尔滨工业大学）；University of Tübingen（图宾根大学）
