We address the problem of vehicle self-localization from multi-modal sensor information and a reference map. The map is generated off-line by extracting landmarks from the vehicle's field of view, while the measurements are collected similarly on the fly. Our goal is to determine the autonomous vehicle's pose from the landmark measurements and map landmarks. To learn this mapping, we propose DeepLocalization, a deep neural network that regresses the vehicle's translation and rotation parameters from unordered and dynamic input landmarks. The proposed network architecture is robust to changes of the dynamic environment and can cope with a small number of extracted landmarks. During the training process we rely on synthetically generated ground-truth. In our experiments, we evaluate two inference approaches in real-world scenarios. We show that DeepLocalization can be combined with regular GPS signals and filtering algorithms such as the extended Kalman filter. Our approach achieves state-of-the-art accuracy and is about ten times faster than the related work.
Due to their ubiquity and long-term stability, pole-like objects are well suited to serve as landmarks for vehicle localization in urban environments. In this work, we present a complete mapping and long-term localization system based on pole landmarks extracted from 3-D lidar data. Our approach features a novel pole detector, a mapping module, and an online localization module, each of which are described in detail, and for which we provide an open-source implementation at www.github.com/acschaefer/polex. In extensive experiments, we demonstrate that our method improves on the state of the ar…
MAN Truck & Bus SE（曼恩卡车与客车公司）；Volkswagen AG（大众汽车公司）
Highly automated driving functions currently often rely on a-priori knowledge from maps for planning and prediction in complex scenarios like cities. This makes map-relative localization an essential skill. In this paper, we address the problem of localization with automotive-grade radars, using a real-time graph-based SLAM approach. The system uses landmarks and odometry information as an abstraction layer. This way, besides radars, all kind of different sensor modalities including cameras and lidars can contribute. A single, semantic landmark map is used and maintained for all sensors. We im…
Jet Propulsion Laboratory（喷气推进实验室）；California Institute of Technology（加州理工学院）
There has been an increase in interest in missions that drive significantly longer distances per day than what has currently been performed. Further, some of these proposed missions require autonomous driving and absolute localization in darkness. For example, the Endurance A mission proposes to drive 1200km of its total traverse at night. The lack of natural light available during such missions limits what can be used as visual landmarks and the range at which landmarks can be observed. In order for planetary rovers to traverse long ranges, onboard absolute localization is critical to the abi…
Vehicle localization in highway tunnels is a challenging issue for autonomous vehicle navigation. Since GPS signals from satellites cannot be received inside a highway tunnel, map-aided localization is essential. However, the environment around the tunnel is composed mostly of an elliptical wall. Thereby, the unique feature points for map matching are few unlike the case outdoors. As a result, it is a very difficult condition to perform vehicle navigation in the tunnel with existing map-aided localization. In this paper, we propose tunnel facility-based precise vehicle localization in highway …
High precision localization is a crucial requirement for the autonomous driving system. Traditional positioning methods have some limitations in providing stable and accurate vehicle poses, especially in an urban environment. Herein, we propose a novel self-localizing method using a monocular camera and a 3D compact semantic map. Pre-collected information of the road landmarks is stored in a self-defined map with a minimal amount of data. We recognize landmarks using a deep neural network, followed with a geometric feature extraction process which promotes the measurement accuracy. The vehicle…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Stanford University（斯坦福大学）；Indian Institute of Technology, Bombay（印度理工学院孟买分校）
For robust GPS-vision navigation in urban areas, we propose an Integrity-driven Landmark Attention (ILA) technique via stochastic reachability. Inspired by cognitive attention in humans, we perform convex optimization to select a subset of landmarks from GPS and vision measurements that maximizes integrity-driven performance. Given known measurement error bounds in non-faulty conditions, our ILA follows a unified approach to address both GPS and vision faults and is compatible with any off-the-shelf estimator. We analyze measurement deviation to estimate the stochastic reachable set of expecte…
Landmark-based robot self-localization has recently garnered interest as a highly-compressive domain-invariant approach for performing visual place recognition (VPR) across domains (e.g., time of day, weather, and season). However, landmark-based self-localization can be an ill-posed problem for a passive observer (e.g., manual robot control), as many viewpoints may not provide an effective landmark view. In this study, we consider an active self-localization task by an active observer and present a novel reinforcement learning (RL)-based next-best-view (NBV) planner. Our contributions are as …
Agricultural robots are expected to increase yields in a sustainable way and automate precision tasks, such as weeding and plant monitoring. At the same time, they move in a continuously changing, semi-structured field environment, in which features can hardly be found and reproduced at a later time. Challenges for Lidar and visual detection systems stem from the fact that plants can be very small, overlapping and have a steadily changing appearance. Therefore, a popular way to localize vehicles with high accuracy is based on ex- pensive global navigation satellite systems and not on natural l…
We present an online landmark selection method for distributed long-term visual localization systems in bandwidth-constrained environments. Sharing a common map for online localization provides a fleet of au- tonomous vehicles with the possibility to maintain and access a consistent map source, and therefore reduce redundancy while increasing efficiency. However, connectivity over a mobile network imposes strict bandwidth constraints and thus the need to minimize the amount of exchanged data. The wide range of varying appearance conditions encountered during long-term visual localization offer…
Fast and Robust Localization for Humanoid Soccer Robot via Iterative Landmark Matching
Robotics and Mechanisms Laboratory (RoMeLa), Department of Mechanical and Aerospace Engineering, University of California, Los Angeles（机器人与机制实验室（RoMeLa）、机械与航空航天工程系、加州大学洛杉矶分校）
Tongji University（同济大学）；Technical University of Munich（慕尼黑工业大学）
In this paper, we present a framework for self-localization of parking robots in a parking lot innovatively using square-like landmarks, aiming to provide a positioning solution with low cost but high accuracy. It utilizes square structures common in parking lots such as pillars, corners or charging piles as robust landmarks and deduces the global pose of the robot in conjunction with an off-line map. The localization is performed in real-time via Particle Filter using a single line scanning LiDAR as main sensor, an odometry as secondary information sources. The system has been tested in a sim…
Pole-centric Descriptors for Robust Robot Localization: Evaluation under Pole-at-Distance (PaD) Observations using the Small Pole Landmark (SPL) Dataset
Fundamental Engineering for Knowledge-Based Society, Graduate School of Engineering, University of Fukui（基于知识社会的基础工程，工程研究生院，福井大学）
Jet Propulsion Laboratory, California Institute of Technology（加州理工学院喷气推进实验室）
Onboard localization capabilities for planetary rovers to date have used relative navigation, by integrating combinations of wheel odometry, visual odometry, and inertial measurements during each drive to track position relative to the start of each drive. At the end of each drive, a ground-in-the-loop (GITL) interaction is used to get a position update from human operators in a more global reference frame, by matching images or local maps from onboard the rover to orbital reconnaissance images or maps of a large region around the rover's current position. Autonomous rover drives are limited i…
Institute of Measurement, Control and Microtechnology, Ulm University（乌尔姆大学测量、控制与微技术研究所）
We present a vehicle self-localization method using point-based deep neural networks. Our approach processes measurements and point features, i.e. landmarks, from a high-definition digital map to infer the vehicle's pose. To learn the best association and incorporate local information between the point sets, we propose an attention mechanism that matches the measurements to the corresponding landmarks. Finally, we use this representation for the point-cloud registration and the subsequent pose regression task. Furthermore, we introduce a training simulation framework that artificially generate…
Center for Intelligent Machines, McGill University（麦吉尔大学智能机器中心）；McGill University（麦吉尔大学）
Visual localization under large changes in scale is an important capability in many robotic mapping applications, such as localizing at low altitudes in maps built at high altitudes, or performing loop closure over long distances. Existing approaches, however, are robust only up to about a 3x difference in scale between map and query images. We propose a novel combination of deep-learning-based object features and state-of-the-art SIFT point-features that yields improved robustness to scale change. This technique is training-free and class-agnostic, and in principle can be deployed in any envi…
Simultaneous Localization and Mapping (SLAM) is one of the key robotics tasks as it tackles simultaneous mapping of the unknown environment defined by multiple landmark positions and localization of the unknown pose (i.e., attitude and position) of the robot in three-dimensional (3D) space. The true SLAM problem is modeled on the Lie group of $\mathbb{SLAM}_{n}\left(3\right)$, and its true dynamics rely on angular and translational velocities. This paper proposes a novel geometric nonlinear stochastic estimator algorithm for SLAM on $\mathbb{SLAM}_{n}\left(3\right)$ that precisely mimics the n…
Dalian Maritime University（大连海事大学）；University of Birmingham（伯明翰大学）；RMIT University（皇家墨尔本理工大学）
Simultaneous localization and mapping (SLAM) using automotive radar sensors can provide enhanced sensing capabilities for autonomous systems. In SLAM applications, with a greater requirement for the environment map, information on the extent of landmarks is vital for precise navigation and path planning. Although object extent estimation has been successfully applied in target tracking, its adaption to SLAM remains unaddressed due to the additional uncertainty of the sensor platform, bias in the odometer reading, as well as the measurement non-linearity. In this paper, we propose to incorporat…
As autonomous systems increasingly rely on onboard sensing for localization and perception, the parallel tasks of motion planning and state estimation become more strongly coupled. This coupling is well-captured by augmenting the planning objective with a posterior-covariance penalty -- however, prediction of the estimator covariance is challenging when the observation model depends on unknown landmarks, as is the case in Simultaneous Localization and Mapping (SLAM). This paper addresses these challenges in the case of landmark- and SLAM-based estimators, enabling efficient prediction (and ult…
ARC Centre of Excellence for Robotic Vision（ARC机器人视觉卓越中心）；Queensland University of Technology (QUT)（昆士兰科技大学）
Research in Simultaneous Localization And Mapping (SLAM) is increasingly moving towards richer world representations involving objects and high level features that enable a semantic model of the world for robots, potentially leading to a more meaningful set of robot-world interactions. Many of these advances are grounded in state-of-the-art computer vision techniques primarily developed in the context of image-based benchmark datasets, leaving several challenges to be addressed in adapting them for use in robotics. In this paper, we derive a formulation for Simultaneous Localization And Mappin…
LiDAR odometry is one of the essential parts of LiDAR simultaneous localization and mapping (SLAM). However, existing LiDAR odometry tends to match a new scan simply iteratively with previous fixed-pose scans, gradually accumulating errors. Furthermore, as an effective joint optimization mechanism, bundle adjustment (BA) cannot be directly introduced into real-time odometry due to the intensive computation of large-scale global landmarks. Therefore, this letter designs a new strategy named a landmark map for bundle adjustment odometry (LMBAO) in LiDAR SLAM to solve these problems. First, BA-ba…
Southern University of Science and Technology（南方科技大学）
Multi-robot localization has been a critical problem for robots performing complex tasks cooperatively. In this paper, we propose a decentralized approach to localize a group of robots in a large featureless environment. The proposed approach only requires that at least one robot remains stationary as a temporary landmark during a certain period of time. The novelty of our approach is threefold: (1) developing a decentralized scheme that each robot calculates their own state and only stores the latest one to reduce storage and computational cost, (2) developing an efficient localization algori…
Dalian Maritime University（大连海事大学）；University of Birmingham（伯明翰大学）；RMIT University（皇家墨尔本理工大学）
This paper focuses on efficient landmark management in radar based simultaneous localization and mapping (SLAM). Landmark management is necessary in order to maintain a consistent map of the estimated landmarks relative to the estimate of the platform's pose. This task is particularly important when faced with multiple detections from the same landmark and/or dynamic environments where the location of a landmark can change. A further challenge with radar data is the presence of false detections. Accordingly, we propose a simple yet efficient rule based solution for radar SLAM landmark manageme…
University of Cincinnati（辛辛那提大学）；Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）；Air Force Research Laboratory（空军研究实验室）
This paper presents a novel data-driven navigation system to navigate an Unmanned Vehicle (UV) in GPS-denied, feature-deficient environments such as tunnels, or mines. The method utilizes landmarks that vehicle can deploy and measure range from to enable localization as the vehicle traverses its pre-defined path through the tunnel. A key question that arises in such scenario is to estimate and reduce the number of landmarks that needs to be deployed for localization before the start of the mission, given some information about the environment. The main focus is to keep the maximum position unc…
VOOM: Robust Visual Object Odometry and Mapping using Hierarchical Landmarks
School of Mechanical Engineering, Beijing Institute of Technology（北京理工大学机械工程学院）；Yangtze Delta Region Academy of Beijing Institute of Technology（北京理工大学长三角研究院）；College of Intelligence Science and Technology, National University of Defense Technology（国防科技大学智能科学学院）
T Stone Robotics Institute, Chinese University of Hong Kong（香港中文大学T石机器人研究所）；Hong Kong Centre for Logistics Robotics（香港物流机器人中心）
Localization in a pre-built map is a basic technique for robot autonomous navigation. Existing mapping and localization methods commonly work well in small-scale environments. As a map grows larger, however, more memory is required and localization becomes inefficient. To solve these problems, map sparsification becomes a practical necessity to acquire a subset of the original map for localization. Previous map sparsification methods add a quadratic term in mixed-integer programming to enforce a uniform distribution of selected landmarks, which requires high memory capacity and heavy computati…
University of Pennsylvania（宾夕法尼亚大学）；Lehigh University（里海大学）
In recent years Landmark Complexes have been successfully employed for localization-free and metric-free autonomous exploration using a group of sensing-limited and communication-limited robots in a GPS-denied environment. To ensure rapid and complete exploration, existing works make assumptions on the density and distribution of landmarks in the environment. These assumptions may be overly restrictive, especially in hazardous environments where landmarks may be destroyed or completely missing. In this paper, we first propose a deep reinforcement learning framework for multi-agent cooperative …
In cooperative localization, communicating mobile agents use inter-agent relative measurements to improve their dead-reckoning-based global localization. Measurement scheduling enables an agent to decide which subset of available inter-agent relative measurements it should process when its computational resources are limited. Optimal measurement scheduling is an NP-hard combinatorial optimization problem. The so-called sequential greedy (SG) algorithm is a popular suboptimal polynomial-time solution for this problem. However, the merit function evaluation for the SG algorithms requires access …
Considering visual localization accuracy at the planning time gives preference to robot motion that can be better localized and thus has the potential of improving vision-based navigation, especially in visually degraded environments. To integrate the knowledge about localization accuracy in motion planning algorithms, a central task is to quantify the amount of information that an image taken at a 6 degree-of-freedom pose brings for localization, which is often represented by the Fisher information. However, computing the Fisher information from a set of sparse landmarks (i.e., a point cloud)…
