A Framework for Depth Estimation and Relative Localization of Ground Robots using Computer Vision
Research Center for Systems and Technologies (SYSTEC)（系统与技术研究中心）；Faculty of Engineering, University of Porto（波尔图大学工程学院）；KTH Royal Institute of Technology（KTH皇家理工学院）
Romulo T. Rodrigues, Pedro Miraldo, Dimos V. Dimarogonas, A. Pedro Aguiar
The 3D depth estimation and relative pose estimation problem within a decentralized architecture is a challenging problem that arises in missions that require coordination among multiple vision-controlled robots. The depth estimation problem aims at recovering the 3D information of the environment. The relative localization problem consists of estimating the relative pose between two robots, by sensing each other's pose or sharing information about the perceived environment. Most solutions for these problems use a set of discrete data without taking into account the chronological order of the events. This paper builds on recent results on continuous estimation to propose a framework that estimates the depth and relative pose between two non-holonomic vehicles. The basic idea consists in estimating the depth of the points by explicitly considering the dynamics of the camera mounted on a ground robot, and feeding the estimates of 3D points observed by both cameras in a filter that computes the relative pose between the robots. We evaluate the convergence for a set of simulated scenarios and show experimental results validating the proposed framework.
We present an unsupervised simultaneous learning framework for the task of monocular camera re-localization and depth estimation from unlabeled video sequences. Monocular camera re-localization refers to the task of estimating the absolute camera pose from an instance image in a known environment, which has been intensively studied for alternative localization in GPS-denied environments. In recent works, camera re-localization methods are trained via supervised learning from pairs of camera images and camera poses. In contrast to previous works, we propose a completely unsupervised learning fr…
Otto von Guericke University Magdeburg（马格德堡奥托·冯·格里克大学）
External localization is an essential part for the indoor operation of small or cost-efficient robots, as they are used, for example, in swarm robotics. We introduce a two-stage localization and instance identification framework for arbitrary robots based on convolutional neural networks. Object detection is performed on an external camera image of the operation zone providing robot bounding boxes for an identification and orientation estimation convolutional neural network. Additionally, we propose a process to generate the necessary training data. The framework was evaluated with 3 different…
Universidad de Chile（智利大学）；Advanced Mining Technology Center（先进采矿技术中心）；Delft University of Technology（代尔夫特理工大学）
We propose a Visual-SLAM based localization and navigation system for service robots. Our system is built on top of the ORB-SLAM monocular system but extended by the inclusion of wheel odometry in the estimation procedures. As a case study, the proposed system is validated using the Pepper robot, whose short-range LIDARs and RGB-D camera do not allow the robot to self-localize in large environments. The localization system is tested in navigation tasks using Pepper in two different environments: a medium-size laboratory, and a large-size hall.
Traditional simultaneous localization and mapping (SLAM) methods focus on improvement in the robot's localization under environment and sensor uncertainty. This paper, however, focuses on mitigating the need for exact localization of a mobile robot to pursue autonomous navigation using a sparse set of images. The proposed method consists of a model architecture - RoomNet, for unsupervised learning resulting in a coarse identification of the environment and a separate local navigation policy for local identification and navigation. The former learns and predicts the scene based on the short ter…
The Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
The core problem of visual multi-robot simultaneous localization and mapping (MR-SLAM) is how to efficiently and accurately perform multi-robot global localization (MR-GL). The difficulties are two-fold. The first is the difficulty of global localization for significant viewpoint difference. Appearance-based localization methods tend to fail under large viewpoint changes. Recently, semantic graphs have been utilized to overcome the viewpoint variation problem. However, the methods are highly time-consuming, especially in large-scale environments. This leads to the second difficulty, which is h…
Zhejiang University（浙江大学）；Alibaba-Zhejiang University Joint Institute of Frontier Technologies（阿里巴巴-浙江大学前沿技术联合研究院）
While visual localization or SLAM has witnessed great progress in past decades, when deploying it on a mobile robot in practice, few works have explicitly considered the kinematic (or dynamic) constraints of the real robotic system when designing state estimators. To promote the practical deployment of current state-of-the-art visual-inertial localization algorithms, in this work we propose a low-cost kinematics-constrained localization system particularly for a skid-steering mobile robot. In particular, we derive in a principle way the robot's kinematic constraints based on the instantaneous …
We demonstrated multi-mobile robot navigation based on Visible Light Positioning(VLP) localization. From our experiment, the VLP can accurately locate robots' positions in navigation.
This paper presents a comprehensive survey on vision-based robotic grasping. We conclude three key tasks during vision-based robotic grasping, which are object localization, object pose estimation and grasp estimation. In detail, the object localization task contains object localization without classification, object detection and object instance segmentation. This task provides the regions of the target object in the input data. The object pose estimation task mainly refers to estimating the 6D object pose and includes correspondence-based methods, template-based methods and voting-based meth…
University of California, Riverside（加州大学河滨分校）；University of Delaware（特拉华大学）
In this paper we present a consistent and distributed state estimator for multi-robot cooperative localization (CL) which efficiently fuses environmental features and loop-closure constraints across time and robots. In particular, we leverage covariance intersection (CI) to allow each robot to only estimate its own state and autocovariance and compensate for the unknown correlations between robots. Two novel multi-robot methods for utilizing common environmental SLAM features are introduced and evaluated in terms of accuracy and efficiency. Moreover, we adapt CI to enable drift-free estimation…
Fast and Robust Localization for Humanoid Soccer Robot via Iterative Landmark Matching
Robotics and Mechanisms Laboratory (RoMeLa), Department of Mechanical and Aerospace Engineering, University of California, Los Angeles（机器人与机制实验室（RoMeLa）、机械与航空航天工程系、加州大学洛杉矶分校）
The University of Michigan - Ann Arbor（密歇根大学安娜堡分校）
In this paper, we introduce a set-theoretic approach for mobile robot localization with infrastructure-based sensing. The proposed method computes sets that over-bound the robot body and orientation under an assumption of known noise bounds on the sensor and robot motion model. We establish theoretical properties and computational approaches for this set-theoretic localization approach and illustrate its application to an automated valet parking example in simulations and to omnidirectional robot localization problems in real-world experiments. We demonstrate that the set-theoretic localizatio…
TECNALIA, Basque Research and Technology Alliance (BRTA)（TECNALIA，巴斯克研究与技术联盟）；University of the Basque Country (UPV-EHU)（巴斯克国家大学）；Robotics and Autonomous Systems Group (RSAIT)（机器人与自主系统组）
Reliability is a key factor for realizing safety guarantee of full autonomous robot systems. In this paper, we focus on reliability in mobile robot localization. Monte Carlo localization (MCL) is widely used for mobile robot localization. However, it is still difficult to guarantee its safety because there are no methods determining reliability for MCL estimate. This paper presents a novel localization framework that enables robust localization, reliability estimation, and quick re-localization, simultaneously. The presented method can be implemented using similar estimation manner to that of …
University of Freiburg（弗莱堡大学）；Toyota Research Institute（丰田研究院）
Indoor localization is one of the crucial enablers for deployment of service robots. Although several successful techniques for indoor localization have been proposed, the majority of them relies on maps generated from data gathered with the same sensor modality used for localization. Typically, tedious labor by experts is needed to acquire this data, thus limiting the readiness of the system as well as its ease of installation for inexperienced operators. In this paper, we propose a memory and computationally efficient monocular camera-based localization system that allows a robot to estimate…
Lafayette College（拉斐特学院）；University of Pennsylvania（宾夕法尼亚大学）
A key limitation of current multi-robot systems is a lack of relative localization, particularly in environments without GPS or motion capture systems. This article presents a centralized method for relatively localizing a 2D swarm using sensors and beacons on the robots themselves. The UKF-based algorithm as well as the requisite novel and cost-effective sensing hardware are discussed. Comparisons with a motion capture system show that the method is capable of localization with errors on the order of the size of the robots.
This paper presents a localization system for mobile robots enabling precise localization in inaccurate building models. The approach leverages local referencing to counteract inherent deviations between as-planned and as-built data for locally accurate registration. We further fuse a novel image-based robust outlier detector with LiDAR data to reject a wide range of outlier measurements from clutter, dynamic objects, and sensor failures. We evaluate the proposed approach on a mobile robot in a challenging real world building construction site. It consistently outperforms the traditional ICP-b…
Universidad de Sevilla（塞维利亚大学）；Universidad Pablo de Olavide（巴勃罗·德·奥拉维德大学）
This paper presents DLL, a fast direct map-based localization technique using 3D LIDAR for its application to aerial robots. DLL implements a point cloud to map registration based on non-linear optimization of the distance of the points and the map, thus not requiring features, neither point correspondences. Given an initial pose, the method is able to track the pose of the robot by refining the predicted pose from odometry. Through benchmarks using real datasets and simulations, we show how the method performs much better than Monte-Carlo localization methods and achieves comparable precision…
Ultra-wideband (UWB) ranging has emerged as a key radio technology for robot positioning and relative localization in multi-robot systems. Multiple works are now advancing towards more scalable systems, but challenges still remain. This paper proposes a novel approach to relative localization in multi-robot systems where the roles of the UWB nodes are dynamically allocated between active nodes (using time-of-flight for ranging estimation to other active nodes) and passive nodes (using time-difference-of-arrival for estimating range differences with respect to pairs of active nodes). We adaptiv…
Relative localization is an important ability for multiple robots to perform cooperative tasks in GPS-denied environment. This paper presents a novel autonomous positioning framework for monocular relative localization of multiple tiny flying robots. This approach does not require any groundtruth data from external systems or manual labelling. Instead, the proposed framework is able to label real-world images with 3D relative positions between robots based on another onboard relative estimation technology, using ultra-wide band (UWB). After training in this self-supervised manner, the proposed…
Technical University of Munich（慕尼黑工业大学）；Sun Yat-sen University（中山大学）
Localization in the environment is an essential navigational capability for animals and mobile robots. In the indoor environment, the global localization problem remains challenging to be perfectly solved with probabilistic methods. However, animals are able to instinctively localize themselves with much less effort. Therefore, it is intriguing and promising to seek biological inspiration from animals. In this paper, we present a biologically-inspired global localization system using a LiDAR sensor that utilizes a hippocampal model and a landmark-based re-localization approach. The experiment …
King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学）
Robots in a swarm take advantage of a motion capture system or GPS sensors to obtain their global position. However, motion capture systems are environment-dependent and GPS sensors are not reliable in occluded environments. For a reliable and versatile operation in a swarm, robots must sense each other and interact locally. Motivated by this requirement, here we propose an on-board localization framework for multi-robot systems. Our framework consists of an anchor robot with three ultrawideband (UWB) sensors and a tag robot with a single UWB sensor. The anchor robot utilizes the three UWB sen…
Having accurate localization capabilities is one of the fundamental requirements of autonomous robots. For underwater vehicles, the choices for effective localization are limited due to limitations of GPS use in water and poor environmental visibility that makes camera-based methods ineffective. Popular inertial navigation methods for underwater localization using Doppler-velocity log sensors, sonar, high-end inertial navigation systems, or acoustic positioning systems require bulky expensive hardware which are incompatible with low cost, bio-inspired underwater robots. In this paper, we intro…
Southern University of Science and Technology（南方科技大学）
Multi-robot localization has been a critical problem for robots performing complex tasks cooperatively. In this paper, we propose a decentralized approach to localize a group of robots in a large featureless environment. The proposed approach only requires that at least one robot remains stationary as a temporary landmark during a certain period of time. The novelty of our approach is threefold: (1) developing a decentralized scheme that each robot calculates their own state and only stores the latest one to reduce storage and computational cost, (2) developing an efficient localization algori…
Most of the existing mobile robot localization solutions are either heavily dependent on pre-installed infrastructures or having difficulty working in highly repetitive environments which do not have sufficient unique features. To address this problem, we propose a magnetic-assisted initialization approach that enhances the performance of infrastructure-free mobile robot localization in repetitive featureless environments. The proposed system adopts a coarse-to-fine structure, which mainly consists of two parts: magnetic field-based matching and laser scan matching. Firstly, the interpolated m…
The University of Michigan（密歇根大学）；University of Washington（华盛顿大学）
Autonomous robot systems for applications from search and rescue to assistive guidance should be able to engage in natural language dialog with people. To study such cooperative communication, we introduce Robot Simultaneous Localization and Mapping with Natural Language (RobotSlang), a benchmark of 169 natural language dialogs between a human Driver controlling a robot and a human Commander providing guidance towards navigation goals. In each trial, the pair first cooperates to localize the robot on a global map visible to the Commander, then the Driver follows Commander instructions to move …
Harbin Institute of Technology Shenzhen Graduate School（哈尔滨工业大学深圳研究生院）；University of Utah（犹他大学）
A new 3D localization and mapping techinque with terrain inclination assistance is proposed in this paper to allow a robot to identify its location and build a global map in an outdoor environment. The Iterative Closest Points (ICP) algorithm and terrain inclination-based localization are combined together to achieve accurate and fast localization and mapping. Inclinations of the terrains the robot navigates are used to achieve local localization during the interval between two laser scans. Using the results of the above localization as the initial condition, the ICP algorithm is then applied …
Continuum Robot Localization using Distributed Time-of-Flight Sensors
University of Toronto Robotics Institute, Canada（多伦多大学机器人研究所，加拿大）；Oxford Robotics Institute, University of Oxford, UK（牛津大学机器人研究所，英国）；Toronto Metropolitan University, Canada（多伦多 Metropolitan 大学，加拿大）
Massachusetts Institute of Technology（麻省理工学院）；Woods Hole Oceanographic Institution（伍兹霍尔海洋研究所）
We present a novel path-planning algorithm to reduce localization error for a network of robots cooperatively localizing via inter-robot range measurements. The quality of localization with range measurements depends on the configuration of the network, and poor configurations can cause substantial localization errors. To reduce the effect of network configuration on localization error for moving networks we consider various optimality measures of the Fisher information matrix (FIM), which have well-studied relationships with the localization error. In particular, we pose a trajectory planning…
A review on cloud robotics based frameworks to solve simultaneous localization and mapping (slam) problem
National Institute of Technology, Raipur（印度国立理工学院赖布尔分校）；Motilal Nehru National Institute of Technology, Allahabad（莫蒂拉尔·尼赫鲁国立理工学院阿拉哈巴德分校）；Indian Institute of Information Technology, Allahabad（印度信息技术学院阿拉哈巴德分校）
University of Luxembourg（卢森堡大学）；Stugalux Construction S.A.（Stugalux建筑股份公司）
Robots in the construction industry can reduce costs through constant monitoring of the work progress, using high precision data capturing. Accurate data capturing requires precise localization of the mobile robot within the environment. In this paper we present our novel work on robot localization which extracts geometric, semantic as well as the topological information from the architectural plans in the form of walls and rooms, and creates the topological and metric-semantic layer of the Situational Graphs (S-Graphs) before navigating in the environment. When the robot navigates in the cons…
University of Pennsylvania（宾夕法尼亚大学）；Drexel University（德雷塞尔大学）
We study the problem that requires a team of robots to perform joint localization and target tracking task while ensuring team connectivity and collision avoidance. The problem can be formalized as a nonlinear, non-convex optimization program, which is typically hard to solve. To this end, we design a two-staged approach that utilizes a greedy algorithm to optimize the joint localization and target tracking performance and applies control barrier functions to ensure safety constraints, i.e., maintaining connectivity of the robot team and preventing inter-robot collisions. Simulated Gazebo expe…
Institute of Systems Science, National University of Singapore（新加坡国立大学系统科学研究所）
In this paper, we propose an novel implementation of a simultaneous localization and mapping (SLAM) system based on a monocular camera from an unmanned aerial vehicle (UAV) using Depth prediction performed with Capsule Networks (CapsNet), which possess improvements over the drawbacks of the more widely-used Convolutional Neural Networks (CNN). An Extended Kalman Filter will assist in estimating the position of the UAV so that we are able to update the belief for the environment. Results will be evaluated on a benchmark dataset to portray the accuracy of our intended approach.
A Survey on Global LiDAR Localization: Challenges, Advances and Open Problems
Hong Kong University of Science and Technology（香港科技大学）；Zhejiang University（浙江大学）；National University of Defense Technology（国防科技大学）；University of Bonn（波恩大学）
This paper mainly studies the localization and mapping of range sensing robots in the confidence-rich map (CRM) and then extends it to provide a full state estimate for information-theoretic exploration. Most previous works about active simultaneous localization and mapping and exploration always assumed the known robot poses or utilized inaccurate information metrics to approximate pose uncertainty, resulting in imbalanced exploration performance and efficiency in the unknown environment. This inspires us to extend the confidence-rich mutual information (CRMI) with measurable pose uncertainty…
Detecting and localizing contacts is essential for robot manipulators to perform contact-rich tasks in unstructured environments. While robot skins can localize contacts on the surface of robot arms, these sensors are not yet robust or easily accessible. As such, prior works have explored using proprioceptive observations, such as joint velocities and torques, to perform contact localization. Many past approaches assume the robot is static during contact incident, a single contact is made at a time, or having access to accurate dynamics models and joint torque sensing. In this work, we relax t…
We present the first fully distributed multi-robot system for dense metric-semantic Simultaneous Localization and Mapping (SLAM). Our system, dubbed Kimera-Multi, is implemented by a team of robots equipped with visual-inertial sensors, and builds a 3D mesh model of the environment in real-time, where each face of the mesh is annotated with a semantic label (e.g., building, road, objects). In Kimera-Multi, each robot builds a local trajectory estimate and a local mesh using Kimera. Then, when two robots are within communication range, they initiate a distributed place recognition and robust po…
