A promising approach to autonomous driving is machine learning. In such systems, training datasets are created that capture the sensory input to a vehicle as well as the desired response. A disadvantage of using a learned navigation system is that the learning process itself may require a huge number of training examples and a large amount of computing. To avoid the need to collect a large training set of driving examples, we describe a system that takes advantage of the huge number of training examples provided by ImageNet, but is able to adapt quickly using a small training set for the specific driving environment.
STONE Dataset: A Scalable Multi-Modal Surround-View 3D Traversability Dataset for Off-Road Robot Navigation
Interdisciplinary Program in Artificial Intelligence, Seoul National University（人工智能交叉学科项目，首尔国立大学）；Department of Electrical and Computer Engineering, Seoul National University（电气与计算机工程系，首尔国立大学）
Massachusetts Institute of Technology（麻省理工学院）；DEVCOM Army Research Laboratory（美国陆军作战能力发展司令部陆军研究实验室）
A key challenge in fast ground robot navigation in 3D terrain is balancing robot speed and safety. Recent work has shown that 2.5D maps (2D representations with additional 3D information) are ideal for real-time safe and fast planning. However, the prevalent approach of generating 2D occupancy grids through raytracing makes the generated map unsafe to plan in, due to inaccurate representation of unknown space. Additionally, existing planners such as MPPI do not consider speeds in known free and unknown space separately, leading to slower overall plans. The RAMP pipeline proposed here solves th…
Learning Traversability for Long Horizon Off-Road Navigation
J. Mike Walker ’66 Department of Mechanical Engineering, Texas A&M University（德克萨斯农工大学J. Mike Walker '66机械工程系）；DEVCOM Army Research laboratory（陆军研究实验室）
Colorado School of Mines（科罗拉多矿业学院）；DEVCOM Army Research Laboratory（DEVCOM陆军研究实验室）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
When robots operate in real-world off-road environments with unstructured terrains, the ability to adapt their navigational policy is critical for effective and safe navigation. However, off-road terrains introduce several challenges to robot navigation, including dynamic obstacles and terrain uncertainty, leading to inefficient traversal or navigation failures. To address these challenges, we introduce a novel approach for adaptation by negotiation that enables a ground robot to adjust its navigational behaviors through a negotiation process. Our approach first learns prediction models for va…
University of British Columbia（不列颠哥伦比亚大学）；Monash University（莫纳什大学）
In this paper, we propose a novel navigation system for mobile robots in pedestrian-rich sidewalk environments. Sidewalks are unique in that the pedestrian-shared space has characteristics of both roads and indoor spaces. Like vehicles on roads, pedestrian movement often manifests as linear flows in opposing directions. On the other hand, pedestrians also form crowds and can exhibit much more random movements than vehicles. Classical algorithms are insufficient for safe navigation around pedestrians and remaining on the sidewalk space. Thus, our approach takes advantage of natural human motion…
ALT-Pilot: Autonomous navigation with Language augmented Topometric maps
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；International Institute of Information Technology, Hyderabad（海得拉巴国际信息技术学院）；Massachusetts Institute of Technology（麻省理工学院）
IIIT Hyderabad（印度海得拉巴国际信息技术学院）；Intel Bangalore（英特尔班加罗尔分公司）；University of California, San Diego（加利福尼亚大学圣迭戈分校）
While several datasets for autonomous navigation have become available in recent years, they tend to focus on structured driving environments. This usually corresponds to well-delineated infrastructure such as lanes, a small number of well-defined categories for traffic participants, low variation in object or background appearance and strict adherence to traffic rules. We propose IDD, a novel dataset for road scene understanding in unstructured environments where the above assumptions are largely not satisfied. It consists of 10,004 images, finely annotated with 34 classes collected from 182 …
Vision-Language Models for Deployable Social Robot Navigation: Bridging Semantic Reasoning and Low-Level Control
Graduate School of Information Science and Technology, Hokkaido University（弘前大学信息科学与技术研究生院）；Graduate School of Information Science and Technology, The University of Tokyo（东京大学信息科学与技术研究生院）
RoadRunner -- Learning Traversability Estimation for Autonomous Off-road Driving
Jet Propulsion Laboratory (JPL)（喷气推进实验室）；California Institute of Technology (Caltech)（加州理工学院）；Swiss Federal Institute of Technology (ETH Zürich)（苏黎世联邦理工学院）
GE Research（通用电气研究院）；Motional（Motional公司）；GE Digital（通用电气数字部门）
Off-road autonomous unmanned ground vehicles (UGVs) are being developed for military and commercial use to deliver crucial supplies in remote locations, help with mapping and surveillance, and to assist war-fighters in contested environments. Due to complexity of the off-road environments and variability in terrain, lighting conditions, diurnal and seasonal changes, the models used to perceive the environment must handle a lot of input variability. Current datasets used to train perception models for off-road autonomous navigation lack of diversity in seasons, locations, semantic classes, as w…
Hong Kong University of Science and Technology（香港科技大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Road curb detection is very important and necessary for autonomous driving because it can improve the safety and robustness of robot navigation in the outdoor environment. In this paper, a novel road curb detection method based on tensor voting is presented. The proposed method processes the dense point cloud acquired using a 3D LiDAR. Firstly, we utilize a sparse tensor voting approach to extract the line and surface features. Then, we use an adaptive height threshold and a surface vector to extract the point clouds of the road curbs. Finally, we utilize the height threshold to segment differ…
Jet Propulsion Laboratory (JPL), California Institute of Technology (Caltech)（加州理工学院喷气推进实验室（JPL））；Swiss Federal Institute of Technology, Zurich（苏黎世联邦理工学院）；Massachusetts Institute of Technology（麻省理工学院）
We propose a framework for resilient autonomous navigation in perceptually challenging unknown environments with mobility-stressing elements such as uneven surfaces with rocks and boulders, steep slopes, negative obstacles like cliffs and holes, and narrow passages. Environments are GPS-denied and perceptually-degraded with variable lighting from dark to lit and obscurants (dust, fog, smoke). Lack of prior maps and degraded communication eliminates the possibility of prior or off-board computation or operator intervention. This necessitates real-time on-board computation using noisy sensor dat…
Long-range indoor navigation requires guiding robots with noisy sensors and controls through cluttered environments along paths that span a variety of buildings. We achieve this with PRM-RL, a hierarchical robot navigation method in which reinforcement learning agents that map noisy sensors to robot controls learn to solve short-range obstacle avoidance tasks, and then sampling-based planners map where these agents can reliably navigate in simulation; these roadmaps and agents are then deployed on robots, guiding them along the shortest path where the agents are likely to succeed. Here we use …
Urban autonomous driving in the presence of pedestrians as vulnerable road users is still a challenging and less examined research problem. This work formulates navigation in urban environments as a multi objective reinforcement learning problem. A deep learning variant of thresholded lexicographic Q-learning is presented for autonomous navigation amongst pedestrians. The multi objective DQN agent is trained on a custom urban environment developed in CARLA simulator. The proposed method is evaluated by comparing it with a single objective DQN variant on known and unknown environments. Evaluati…
Autonomous navigation for low-altitude UAVs in urban areas
Survey Copter / Airbus Defense and Space（空客防务与航天公司Survey Copter部门）；Hubert Curien Laboratory（于贝尔·居里安实验室）；Center for Research in Computer Vision, UCF（中佛罗里达大学计算机视觉研究中心）
Nuro, Inc.（Nuro公司）；University of Pennsylvania（宾夕法尼亚大学）
Autonomous navigation in crowded, complex urban environments requires interacting with other agents on the road. A common solution to this problem is to use a prediction model to guess the likely future actions of other agents. While this is reasonable, it leads to overly conservative plans because it does not explicitly model the mutual influence of the actions of interacting agents. This paper builds a reinforcement learning-based method named MIDAS where an ego-agent learns to affect the control actions of other cars in urban driving scenarios. MIDAS uses an attention-mechanism to handle an…
Reinforcement learning can enable robots to navigate to distant goals while optimizing user-specified reward functions, including preferences for following lanes, staying on paved paths, or avoiding freshly mowed grass. However, online learning from trial-and-error for real-world robots is logistically challenging, and methods that instead can utilize existing datasets of robotic navigation data could be significantly more scalable and enable broader generalization. In this paper, we present ReViND, the first offline RL system for robotic navigation that can leverage previously collected data …
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Sony AI（索尼人工智能）
This paper presents a learning-based approach to consider the effect of unobservable world states in kinodynamic motion planning in order to enable accurate high-speed off-road navigation on unstructured terrain. Existing kinodynamic motion planners either operate in structured and homogeneous environments and thus do not need to explicitly account for terrain-vehicle interaction, or assume a set of discrete terrain classes. However, when operating on unstructured terrain, especially at high speeds, even small variations in the environment will be magnified and cause inaccurate plan execution.…
Carnegie Mellon University（卡内基梅隆大学）；Indian Institute of Technology Kharagpur（印度理工学院卡拉格普尔分校）
The process of designing costmaps for off-road driving tasks is often a challenging and engineering-intensive task. Recent work in costmap design for off-road driving focuses on training deep neural networks to predict costmaps from sensory observations using corpora of expert driving data. However, such approaches are generally subject to over-confident mispredictions and are rarely evaluated in-the-loop on physical hardware. We present an inverse reinforcement learning-based method of efficiently training deep cost functions that are uncertainty-aware. We do so by leveraging recent advances …
Evaluation of Safety Constraints in Autonomous Navigation with Deep Reinforcement Learning
Moscow Institute of Physics and Technology（莫斯科物理技术学院）；AIRI（AIRI（人工智能研究所））；Federal Research Center for Computer Science and Control of Russian Academy of Sciences（俄罗斯科学院计算机科学与控制联邦研究中心）
Topological mapping for traversability-aware long-range navigation in off-road terrain
Center for Intelligent Machines, McGill University（麦吉尔大学智能机器中心）；Gault Nature Reserve（戈尔特自然保护区）；CMLabs（CMLabs公司）；National Canadian Robotics Network（加拿大国家机器人网络）
Colorado School of Mines（科罗拉多矿业学院）；US Army Research Laboratory（美国陆军研究实验室）
Terrain adaptation is an essential capability for a ground robot to effectively traverse unstructured off-road terrain in real-world field environments such as forests. However, the expected robot behaviors generated by terrain adaptation methods cannot always be executed accurately due to setbacks such as wheel slip and reduced tire pressure. To address this problem, we propose a novel approach for consistent behavior generation that enables the ground robot's actual behaviors to more accurately match expected behaviors while adapting to a variety of unstructured off-road terrain. Our approac…
Universidade de Pernambuco（伯南布哥大学）；Istituto Italiano di Tecnologia（意大利理工学院）
Dealing with social tasks in robotic scenarios is difficult, as having humans in the learning loop is incompatible with most of the state-of-the-art machine learning algorithms. This is the case when exploring Incremental learning models, in particular the ones involving reinforcement learning. In this work, we discuss this problem and possible solutions by analysing a previous study on adaptive convolutional encoders for a social navigation task.
Ulsan National Institute of Science and Technology（蔚山国立科学技术院）
This paper presents a safe, efficient, and agile ground vehicle navigation algorithm for 3D off-road terrain environments. Off-road navigation is subject to uncertain vehicle-terrain interactions caused by different terrain conditions on top of 3D terrain topology. The existing works are limited to adopt overly simplified vehicle-terrain models. The proposed algorithm learns the terrain-induced uncertainties from driving data and encodes the learned uncertainty distribution into the traversability cost for path evaluation. The navigation path is then designed to optimize the uncertainty-aware …
