For the safe and successful navigation of autonomous vehicles in unstructured environments, the traversability of terrain should vary based on the driving capabilities of the vehicles. Actual driving experience can be utilized in a self-supervised fashion to learn vehicle-specific traversability. However, existing methods for learning self-supervised traversability are not highly scalable for learning the traversability of various vehicles. In this work, we introduce a scalable framework for learning self-supervised traversability, which can learn the traversability directly from vehicle-terrain interaction without any human supervision. We train a neural network that predicts the proprioceptive experience that a vehicle would undergo from 3D point clouds. Using a novel PU learning method, the network simultaneously identifies non-traversable regions where estimations can be overconfident. With driving data of various vehicles gathered from simulation and the real world, we show that our framework is capable of learning the self-supervised traversability of various vehicles. By integrating our framework with a model predictive controller, we demonstrate that estimated traversability results in effective navigation that enables distinct maneuvers based on the driving characteristics of the vehicles. In addition, experimental results validate the ability of our method to identify and avoid non-traversable regions.
University of Surrey（萨里大学）；UK Atomic Energy Authority（英国原子能管理局）
Terrain traversability analysis plays a major role in ensuring safe robotic navigation in unstructured environments. However, real-time constraints frequently limit the accuracy of online tests especially in scenarios where realistic robot-terrain interactions are complex to model. In this context, we propose a deep learning framework trained in an end-to-end fashion from elevation maps and trajectories to estimate the occurrence of failure events. The network is first trained and tested in simulation over synthetic maps generated by the OpenSimplex algorithm. The prediction performance of the…
Georgia Institute of Technology（佐治亚理工学院）；NASA Jet Propulsion Laboratory, California Institute of Technology（美国国家航空航天局喷气推进实验室，加州理工学院）；University of Goettingen（哥廷根大学）
One of the main challenges in autonomous robotic exploration and navigation in unknown and unstructured environments is determining where the robot can or cannot safely move. A significant source of difficulty in this determination arises from stochasticity and uncertainty, coming from localization error, sensor sparsity and noise, difficult-to-model robot-ground interactions, and disturbances to the motion of the vehicle. Classical approaches to this problem rely on geometric analysis of the surrounding terrain, which can be prone to modeling errors and can be computationally expensive. Moreo…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；EarthSense Inc.（EarthSense公司）；DEVCOM Army Research Lab（DEVCOM陆军研究实验室）
We present a self-supervised approach for learning to predict traversable paths for wheeled mobile robots that require good traction to navigate. Our algorithm, termed WayFAST (Waypoint Free Autonomous Systems for Traversability), uses RGB and depth data, along with navigation experience, to autonomously generate traversable paths in outdoor unstructured environments. Our key inspiration is that traction can be estimated for rolling robots using kinodynamic models. Using traction estimates provided by an online receding horizon estimator, we are able to train a traversability prediction neural…
3DTTNet: Multimodal Fusion-Based 3D Traversable Terrain Modeling for Off-Road Environments
Shenzhen Automotive Research Institute, Beijing Institute of Technology, Shenzhen（深圳汽车研究院，北京理工大学，深圳）；School of Mechanical Engineering, Beijing Institute of Technology, Beijing（机械工程学院，北京理工大学，北京）；Research Center for Intelligent Computing Systems, Institute of Computing Technology, Chinese Academy of Sciences, Beijing（智能计算系统研究中心，计算技术研究所，中国科学院，北京）
Heterogeneous Robot Collaboration in Unstructured Environments with Grounded Generative Intelligence
Dept. of Electrical and Computer Engineering at Texas A&M University（德克萨斯A&M大学电气与计算机工程系）；GRASP Laboratory at the University of Pennsylvania（宾夕法尼亚大学GRASP实验室）
Baidu Research（百度研究院）；University of Maryland, College Park（马里兰大学帕克分校）
We present a terrain traversability mapping and navigation system (TNS) for autonomous excavator applications in an unstructured environment. We use an efficient approach to extract terrain features from RGB images and 3D point clouds and incorporate them into a global map for planning and navigation. Our system can adapt to changing environments and update the terrain information in real-time. Moreover, we present a novel dataset, the Complex Worksite Terrain (CWT) dataset, which consists of RGB images from construction sites with seven categories based on navigability. Our novel algorithms i…
ZeST: A VLM-based Zero-Shot Traversability Navigation for Unknown Environments
Field Robotics Engineering and Science Hub (FRESH), Illinois Autonomous Farm, University of Illinois at Urbana-Champaign (UIUC), IL（伊利诺伊大学厄巴纳-香槟分校）；Mobile Robotics Group, São Carlos School of Engineering, University of São Paulo (EESC-USP), São Carlos, SP, Brazil（圣保罗大学）
Dual-BEV Nav: Dual-layer BEV-based Heuristic Path Planning for Robotic Navigation in Unstructured Outdoor Environments
Software Engineering Institute, East China Normal University（华东师范大学软件工程学院）；School of Geospatial Information, Information Engineering University（信息工程大学地理空间信息学院）；College of Computer and Cyber Security, Fujian Normal University（福建师范大学计算机与网络空间安全学院）
针对非结构化室外环境路径规划难题，提出 Dual-BEV Nav 方法，通过整合局部与全局 BEV 可通行性建立双层启发式规划范式，显著提升了时间距离预测精度并实现了长距离室外导航。
Trinity: Unifying Class-Agnostic Terrain and Semantic Segmentation for Unstructured Outdoor Environments by Leveraging Synthetic Data
Institute of Robotics and Mechatronics, German Aerospace Center (DLR)（机器人与机电系统研究所，德国航空航天中心（DLR））；Federal Institute of Technology Zurich (ETH Zurich)（苏黎世联邦理工学院（ETH Zurich））；Robotics and AI Institute (RAI)（机器人与人工智能研究所（RAI））
University of Tokyo（东京大学）；California Institute of Technology（加州理工学院）；Jet Propulsion Laboratory, California Institute of Technology（加州理工学院喷气推进实验室）
Although ground robotic autonomy has gained widespread usage in structured and controlled environments, autonomy in unknown and off-road terrain remains a difficult problem. Extreme, off-road, and unstructured environments such as undeveloped wilderness, caves, and rubble pose unique and challenging problems for autonomous navigation. To tackle these problems we propose an approach for assessing traversability and planning a safe, feasible, and fast trajectory in real-time. Our approach, which we name STEP (Stochastic Traversability Evaluation and Planning), relies on: 1) rapid uncertainty-awa…
Dalle Molle Institute for Artificial Intelligence (IDSIA), USI-SUPSI（达勒·莫勒人工智能研究所（IDSIA），瑞士意大利语区大学-南方应用科技大学）
Mobile ground robots operating on unstructured terrain must predict which areas of the environment they are able to pass in order to plan feasible paths. We address traversability estimation as a heightmap classification problem: we build a convolutional neural network that, given an image representing the heightmap of a terrain patch, predicts whether the robot will be able to traverse such patch from left to right. The classifier is trained for a specific robot model (wheeled, tracked, legged, snake-like) using simulation data on procedurally generated training terrains; the trained classifi…
Continual Learning for Traversability Prediction with Uncertainty-Aware Adaptation
Ulsan National Institute of Science and Technology（蔚山国立科学技术院）；Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University of Munich (TUM)（慕尼黑机器人与机器智能研究所（MIRMI），慕尼黑工业大学（TUM））
We propose a novel method for autonomous legged robot navigation in densely vegetated environments with a variety of pliable/traversable and non-pliable/untraversable vegetation. We present a novel few-shot learning classifier that can be trained on a few hundred RGB images to differentiate flora that can be navigated through, from the ones that must be circumvented. Using the vegetation classification and 2D lidar scans, our method constructs a vegetation-aware traversability cost map that accurately represents the pliable and non-pliable obstacles with lower, and higher traversability costs,…
RoadRunner -- Learning Traversability Estimation for Autonomous Off-road Driving
Jet Propulsion Laboratory (JPL)（喷气推进实验室）；California Institute of Technology (Caltech)（加州理工学院）；Swiss Federal Institute of Technology (ETH Zürich)（苏黎世联邦理工学院）
WayFASTER: a Self-Supervised Traversability Prediction for Increased Navigation Awareness
University of Illinois at Urbana-Champaign (UIUC)（伊利诺伊大学厄巴纳-香槟分校）；Field Robotics Engineering and Science Hub (FRESH)（野外机器人工程与科学中心）；Illinois Autonomous Farm（伊利诺伊自主农场）
Watch Your Step: Learning Semantically-Guided Locomotion in Cluttered Environment
School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）；School of Mechanical and Mining Engineering, The University of Queensland（昆士兰大学机械与采矿工程学院）
PRISM: Multimodal Terrain Mapping for Rover Navigation in Unstructured Environments
SnT, University of Luxembourg（卢森堡大学安全、可靠性和信任跨学科中心）；Department of Automation and Systems Engineering, Universidad de Málaga, Andalucía Tech（马拉加大学自动化与系统工程系，安达卢西亚技术大学）
KiteRunner: Language-Driven Cooperative Local-Global Navigation Policy with UAV Mapping in Outdoor Environments
Software Engineering Institute, East China Normal University（华东师范大学软件工程学院）；School of Geospatial Information, Information Engineering University（信息工程大学地理空间信息学院）；Institute of Machine Intelligence (IMI), University of Shanghai for Science and Technology（上海理工大学机器智能研究院）；School of Software, Liaoning Technical University（辽宁工程技术大学软件学院）；Fujian (Quanzhou) Institute of Advanced Manufacturing Technology（福建（泉州）先进制造技术研究院）
Fast and accurate path planning is important for ground robots to achieve safe and efficient autonomous navigation in unstructured outdoor environments. However, most existing methods exploiting either 2D or 2.5D maps struggle to balance the efficiency and safety for ground robots navigating in such challenging scenarios. In this paper, we propose a novel hybrid map representation by fusing a 2D grid and a 2.5D digital elevation map. Based on it, a novel path planning method is proposed, which considers the robot poses during traversability estimation. By doing so, our method explicitly takes …
Tsinghua University（清华大学）；Harbin Institute of Technology（哈尔滨工业大学）；Southern University of Science and Technology（南方科技大学）
Autonomous navigation of ground robots has been widely used in indoor structured 2D environments, but there are still many challenges in outdoor 3D unstructured environments, especially in rough, uneven terrains. This paper proposed a plane-fitting based uneven terrain navigation framework (PUTN) to solve this problem. The implementation of PUTN is divided into three steps. First, based on Rapidly-exploring Random Trees (RRT), an improved sample-based algorithm called Plane Fitting RRT* (PF-RRT*) is proposed to obtain a sparse trajectory. Each sampling point corresponds to a custom traversabil…
CLEAR: A Semantic-Geometric Terrain Abstraction for Large-Scale Unstructured Environments
Department of Computer Science and Engineering, University at Buffalo（计算机科学与工程系，布法罗大学）；Department of Mechanical and Aerospace Engineering, University at Buffalo（机械与航空航天工程系，布法罗大学）
