Fully Onboard SLAM for Distributed Mapping with a Swarm of Nano-Drones
Department of Information Technology and Electrical Engineering (D-ITET), ETH Zürich（信息科技与电气工程系（D-ITET），苏黎世联邦理工学院）；Department of Electrical, Electronic and Information Engineering (DEI), University of Bologna（电气、电子与信息工程系（DEI），博洛尼亚大学）
Carl Friess, Vlad Niculescu, Tommaso Polonelli, Michele Magno, Luca Benini
无人飞行器（UAV）的应用正迅速增加，涵盖从监视和急救任务到涉及与其他机器或人类协作的工业自动化。为最大化区域覆盖并减少任务延迟，协作无人机集群已成为重要的研究方向。然而，该方法需要解决在定位、建图和通信方面的开放挑战。本研究描述了一个基于纳型无人机集群的分布式建图系统，其特征是有效载荷限制为 35 g，且机载传感和计算能力受到严格限制。每个纳型无人机配备四个 64 像素深度传感器，用于测量四个方向上到障碍物的相对距离。所提出的系统融合来自集群的信息，并在不依赖任何外部基础设施的情况下生成连贯的栅格地图。数据融合使用迭代最近点算法和基于图的同步定位与建图算法执行，完全在无人机的低功耗 ARM Cortex-M 微控制器上运行，仅占用 192 kB 的内存。在三个不同迷宫中收集的实地结果（使用多达 4 架纳型无人机的集群）证明了 12 cm 的建图精度，并证明建图时间与智能体数量成反比。所提出的框架在通信带宽和机载计算复杂度方面呈线性扩展，支持多达 20 架纳型无人机之间的通信，并在所选配置下仅需 50 kB 内存即可支持高达 180 m² 区域的建图。
英文摘要：
The use of Unmanned Aerial Vehicles (UAVs) is rapidly increasing in applications ranging from surveillance and first-aid missions to industrial automation involving cooperation with other machines or humans. To maximize area coverage and reduce mission latency, swarms of collaborating drones have become a significant research direction. However, this approach requires open challenges in positioning, mapping, and communications to be addressed. This work describes a distributed mapping system based on a swarm of nano-UAVs, characterized by a limited payload of 35 g and tightly constrained onboard sensing and computing capabilities. Each nano-UAV is equipped with four 64-pixel depth sensors that measure the relative distance to obstacles in four directions. The proposed system merges the information from the swarm and generates a coherent grid map without relying on any external infrastructure. The data fusion is performed using the iterative closest point algorithm and a graph-based simultaneous localization and mapping algorithm, running entirely onboard the UAV's low-power ARM Cortex-M microcontroller with just 192 kB of memory. Field results gathered in three different mazes with a swarm of up to 4 nano-UAVs prove a mapping accuracy of 12 cm and demonstrate that the mapping time is inversely proportional to the number of agents. The proposed framework scales linearly in terms of communication bandwidth and onboard computational complexity, supporting communication between up to 20 nano-UAVs and mapping of areas up to 180 m2 with the chosen configuration requiring only 50 kB of memory.
KU Leuven（鲁汶大学）；imec（比利时微电子研究中心）；Ghent University（根特大学）；VUB（布鲁塞尔自由大学）
Learning to safely navigate in unknown environments is an important task for autonomous drones used in surveillance and rescue operations. In recent years, a number of learning-based Simultaneous Localisation and Mapping (SLAM) systems relying on deep neural networks (DNNs) have been proposed for applications where conventional feature descriptors do not perform well. However, such learning-based SLAM systems rely on DNN feature encoders trained offline in typical deep learning settings. This makes them less suited for drones deployed in environments unseen during training, where continual ada…
Quadrupeds are robots that have been of interest in the past few years due to their versatility in navigating across various terrain and utility in several applications. For quadrupeds to navigate without a predefined map a priori, they must rely on SLAM approaches to localize and build the map of the environment. Despite the surge of interest and research development in SLAM and quadrupeds, there still has yet to be an open-source package that capitalizes on the onboard sensors of an affordable quadruped. This motivates the A1 SLAM package, which is an open-source ROS package that provides th…
AERO-VIS: Asynchronous Event-based Real-time Onboard Visual-Inertial SLAM
Mobile Robotics Lab, ETH Zürich（苏黎世联邦理工学院移动机器人实验室）；Technical University of Munich（慕尼黑技术大学）；Munich Institute of Robotics and Machine Intelligence (MIRMI)（慕尼黑机器人与机器智能研究所）
NanoSLAM: Enabling Fully Onboard SLAM for Tiny Robots
Integrated Systems Laboratory of ETH Zürich（苏黎世联邦理工学院集成系统实验室）；Center for Project-Based Learning of ETH Zürich（苏黎世联邦理工学院项目导向学习中心）；Department of Electrical, Electronic and Information Engineering of University of Bologna（博洛尼亚大学电子、电子与信息工程系）
Technische Universität München（慕尼黑工业大学）；Oculus Research（奥库鲁斯研究院）；RWTH Aachen University（亚琛工业大学）
Micro aerial vehicles (MAVs) are strongly limited in their payload and power capacity. In order to implement autonomous navigation, algorithms are therefore desirable that use sensory equipment that is as small, low-weight, and low-power consuming as possible. In this paper, we propose a method for autonomous MAV navigation and exploration using a low-cost consumer-grade quadrocopter equipped with a monocular camera. Our vision-based navigation system builds on LSD-SLAM which estimates the MAV trajectory and a semi-dense reconstruction of the environment in real-time. Since LSD-SLAM only deter…
Scalable Outdoors Autonomous Drone Flight with Visual-Inertial SLAM and Dense Submaps Built without LiDAR
Technical University of Munich（技术大学慕尼黑）；School of Computation, Information and Technology（计算、信息与技术学院）；Mobile Robotics Lab（移动机器人实验室）；Department of Computing（计算学院）；Imperial College London（伦敦帝国学院）；Department of Mechanical and Process Engineering（机械与过程工程学院）；ETH Zurich（苏黎世联邦理工学院）；Munich Institute of Robotics and Machine Intelligence (MIRMI)（慕尼黑机器人与机器智能研究所）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）
Simultaneous Localization and 3D-Semi Dense Mapping for Micro Drones Using Monocular Camera and Inertial Sensors
The Hatter Department of Marine Technologies, University of Haifa, 3498838, Israel（海洋技术系，海法大学）；Computer Science, University of Haifa, 3498838, Israel（计算机科学，海法大学）
In this paper, we propose a tightly-coupled, multi-modal simultaneous localization and mapping (SLAM) framework, integrating an extensive set of sensors: IMU, cameras, multiple lidars, and Ultra-wideband (UWB) range measurements, hence referred to as VIRAL (visual-inertial-ranging-lidar) SLAM. To achieve such a comprehensive sensor fusion system, one has to tackle several challenges such as data synchronization, multi-threading programming, bundle adjustment (BA), and conflicting coordinate frames between UWB and the onboard sensors, so as to ensure real-time localization and smooth updates in…
UAV-assisted Visual SLAM Generating Reconstructed 3D Scene Graphs in GPS-denied Environments
University of Luxembourg（卢森堡大学）；Interdisciplinary Centre for Security, Reliability, and Trust (SnT)（跨学科安全、可靠性与信任中心（SnT））；Faculty of Science, Technology, and Medicine, University of Luxembourg（卢森堡大学科学、技术与医学学院）
Control-Barrier-Aided Teleoperation with Visual-Inertial SLAM for Safe MAV Navigation in Complex Environments
Learning Systems and Robotics Lab, School of Computation, Information and Technology, Technical University of Munich（学习系统与机器人实验室，计算、信息与技术学院，慕尼黑技术大学）；Smart Robotics Lab, School of Computation, Information and Technology, Technical University of Munich（智能机器人实验室，计算、信息与技术学院，慕尼黑技术大学）；Smart Robotics Lab, Department of Computing, Imperial College London（智能机器人实验室，计算系，伦敦帝国理工学院）；Munich Institute of Robotics and Machine Intellig（慕尼黑机器人与机器智能研究所）
Simultaneous Localization And Mapping (SLAM) is a task to estimate the robot location and to reconstruct the environment based on observation from sensors such as LIght Detection And Ranging (LiDAR) and camera. It is widely used in robotic applications such as autonomous driving and drone delivery. Traditional LiDAR-based SLAM algorithms mainly leverage the geometric features from the scene context, while the intensity information from LiDAR is ignored. Some recent deep-learning-based SLAM algorithms consider intensity features and train the pose estimation network in an end-to-end manner. How…
PiDrone is a quadrotor platform created to accompany an introductory robotics course. Students build an autonomous flying robot from scratch and learn to program it through assignments and projects. Existing educational robots do not have significant autonomous capabilities, such as high-level planning and mapping. We present a hardware and software framework for an autonomous aerial robot, in which all software for autonomy can run onboard the drone, implemented in Python. We present an Unscented Kalman Filter (UKF) for accurate state estimation. Next, we present an implementation of Monte Ca…
Visual Simultaneous Localization and Mapping (v-SLAM) and navigation of multirotor Unmanned Aerial Vehicles (UAV) in an unknown environment have grown in popularity for both research and education. However, due to the complex hardware setup, safety precautions, and battery constraints, extensive physical testing can be expensive and time-consuming. As an alternative solution, simulation tools lower the barrier to carry out the algorithm testing and validation before field trials. In this letter, we customize the ROS-Gazebo-PX4 simulator in deep and provide an end-to-end simulation solution for…
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）
Existing Simultaneous Localization and Mapping (SLAM) approaches are limited in their scalability due to growing map size in long-term robot operation. Moreover, processing such maps for localization and planning tasks leads to the increased computational resources required onboard. To address the problem of memory consumption in long-term operation, we develop a novel real-time SLAM algorithm, MeSLAM, that is based on neural field implicit map representation. It combines the proposed global mapping strategy, including neural networks distribution and region tracking, with an external odometry…
Visual inertial odometry and SLAM algorithms are widely used in various fields, such as service robots, drones, and autonomous vehicles. Most of the SLAM algorithms are based on assumption that landmarks are static. However, in the real-world, various dynamic objects exist, and they degrade the pose estimation accuracy. In addition, temporarily static objects, which are static during observation but move when they are out of sight, trigger false positive loop closings. To overcome these problems, we propose a novel visual-inertial SLAM framework, called DynaVINS, which is robust against both d…
This paper presents ORB-SLAM3, the first system able to perform visual, visual-inertial and multi-map SLAM with monocular, stereo and RGB-D cameras, using pin-hole and fisheye lens models. The first main novelty is a feature-based tightly-integrated visual-inertial SLAM system that fully relies on Maximum-a-Posteriori (MAP) estimation, even during the IMU initialization phase. The result is a system that operates robustly in real-time, in small and large, indoor and outdoor environments, and is 2 to 5 times more accurate than previous approaches. The second main novelty is a multiple map syste…
In this paper, we present a deep learning-based network, GCNv2, for generation of keypoints and descriptors. GCNv2 is built on our previous method, GCN, a network trained for 3D projective geometry. GCNv2 is designed with a binary descriptor vector as the ORB feature so that it can easily replace ORB in systems such as ORB-SLAM2. GCNv2 significantly improves the computational efficiency over GCN that was only able to run on desktop hardware. We show how a modified version of ORB-SLAM2 using GCNv2 features runs on a Jetson TX2, an embedded low-power platform. Experimental results show that GCNv…
Instituto de Investigación en Ingeniería de Aragón (I3A), Universidad de Zaragoza（萨拉戈萨大学阿拉贡工程研究所（I3A））；Oculus Research
We present ORB-SLAM2 a complete SLAM system for monocular, stereo and RGB-D cameras, including map reuse, loop closing and relocalization capabilities. The system works in real-time on standard CPUs in a wide variety of environments from small hand-held indoors sequences, to drones flying in industrial environments and cars driving around a city. Our back-end based on bundle adjustment with monocular and stereo observations allows for accurate trajectory estimation with metric scale. Our system includes a lightweight localization mode that leverages visual odometry tracks for unmapped regions …
SKiD-SLAM: Robust, Lightweight, and Distributed Multi-Robot LiDAR SLAM in Resource-Constrained Field Environments
Dept. Electr. and Comput. Eng.（电子与计算机工程系）；Inha University（inha大学）；Laboratory for Information & Decision Systems (LIDS)（信息与决策系统实验室）；Massachusetts Institute of Technology（麻省理工学院）
Centre for Autonomous Systems（自主系统中心）；University of Technology, Sydney（悉尼科技大学）
In this paper, we study the back-end of simultaneous localization and mapping (SLAM) problem in deforming environment, where robot localizes itself and tracks multiple non-rigid soft surface using its onboard sensor measurements. An elaborate analysis is conducted on conventional deformation modelling method, Embedded Deformation (ED) graph. We demonstrate and prove that the ED graph widely used in such scenarios is unobservable and leads to multiple solutions unless suitable priors are provided. Example as well as theoretical prove are provided to show the ambiguity of ED graph and camera pos…
When adapting Simultaneous Mapping and Localization (SLAM) to real-world applications, such as autonomous vehicles, drones, and augmented reality devices, its memory footprint and computing cost are the two main factors limiting the performance and the range of applications. In sparse feature based SLAM algorithms, one efficient way for this problem is to limit the map point size by selecting the points potentially useful for local and global bundle adjustment (BA). This study proposes an efficient graph optimization for sparsifying map points in SLAM systems. Specifically, we formulate a maxi…
Delft University of Technology（代尔夫特理工大学）；Micro Air Vehicle lab, Control and Simulation, Delft University of Technology（代尔夫特理工大学控制与仿真系微型飞行器实验室）
Drone racing is becoming a popular sport where human pilots have to control their drones to fly at high speed through complex environments and pass a number of gates in a pre-defined sequence. In this paper, we develop an autonomous system for drones to race fully autonomously using only onboard resources. Instead of commonly used visual navigation methods, such as simultaneous localization and mapping and visual inertial odometry, which are computationally expensive for micro aerial vehicles (MAVs), we developed the highly efficient snake gate detection algorithm for visual navigation, which …
NANO-SLAM : Natural Gradient Gaussian Approximation for Vehicle SLAM
School of Vehicle and Mobility, Tsinghua University（车辆与移动性学院，清华大学）；Department of Advanced Manufacturing and Robotics, College of Engineering, Peking University（先进制造与机器人系，工程学院，北京大学）；School of Vehicle and Mobility and College of Artificial Intelligence, Tsinghua University（车辆与移动性学院和人工智能学院，清华大学）
针对车辆SLAM中现有RBPF方法因线性化引入近似误差的问题，提出NANO-SLAM，通过自然梯度下降求解高斯参数优化问题来优化采样分布，在Sydney Victoria Park数据集上定位精度提升超50%且计算成本极低。
Coordinate-Consistent Localization via Continuous-Time Calibration and Fusion of UWB and SLAM Observations
Faculty of Electrical；Electronic Engineering Ho Chi Minh City University of Technology, VNU-HCM Ho Chi Minh City, Vietnam；School of Electrical；Electronic Engineering Nanyang Technological University Singapore；Electronic Engineering Ho Chi Minh City University of Technology, VNU-HCM Ho Chi Minh City, Vietnam (Corresponding author)
SlideSLAM: Sparse, Lightweight, Decentralized Metric-Semantic SLAM for Multi-Robot Navigation
Microsoft（微软公司）；GRASP Laboratory, University of Pennsylvania（宾夕法尼亚大学GRASP实验室）；University of Pennsylvania（宾夕法尼亚大学）；Department of Electrical and Computer Engineering, University of California San Diego（加州大学圣地亚哥分校电气与计算机工程系）
SLAM-Based Navigation and Fault Resilience in a Surveillance Quadcopter with Embedded Vision Systems
Guru Gobind Singh Indraprastha University（古鲁·戈宾德·辛格·因德拉普拉斯塔大学）；Delhi Skill and Entrepreneurship University (DSEU)（德里技能与创业大学（DSEU））；Delhi Institute of Tool Engineering（德里工具工程学院）
The fusion of camera sensor and inertial data is a leading method for ego-motion tracking in autonomous and smart devices. State estimation techniques that rely on non-linear filtering are a strong paradigm for solving the associated information fusion task. The de facto inference method in this space is the celebrated extended Kalman filter (EKF), which relies on first-order linearizations of both the dynamical and measurement model. This paper takes a critical look at the practical implications and limitations posed by the EKF, especially under faulty visual feature associations and the pres…
KU Leuven（鲁汶大学）；imec（比利时微电子研究中心）；Gent University（根特大学）；VUB（布鲁塞尔自由大学）
This work proposes a first-of-its-kind SLAM architecture fusing an event-based camera and a Frequency Modulated Continuous Wave (FMCW) radar for drone navigation. Each sensor is processed by a bio-inspired Spiking Neural Network (SNN) with continual Spike-Timing-Dependent Plasticity (STDP) learning, as observed in the brain. In contrast to most learning-based SLAM systems%, which a) require the acquisition of a representative dataset of the environment in which navigation must be performed and b) require an off-line training phase, our method does not require any offline training phase, but ra…
Collaborative SLAM enables a group of agents to simultaneously co-localize and jointly map an environment, thus paving the way to wide-ranging applications of multi-robot perception and multi-user AR experiences by eliminating the need for external infrastructure or pre-built maps. This article presents COVINS, a novel collaborative SLAM system, that enables multi-agent, scalable SLAM in large environments and for large teams of more than 10 agents. The paradigm here is that each agent runs visual-inertial odomety independently onboard in order to ensure its autonomy, while sharing map informa…
Simultaneous localization and mapping (SLAM) is used to predict the dynamic motion path of a moving platform based on the location coordinates and the precise mapping of the physical environment. SLAM has great potential in augmented reality (AR), autonomous vehicles, viz. self-driving cars, drones, Autonomous navigation robots (ANR). This work introduces a hybrid learning model that explores beyond feature fusion and conducts a multimodal weight sewing strategy towards improving the performance of a baseline SLAM algorithm. It carries out weight enhancement of the front end feature extractor …
RootSLAM: Square Root Bundle Adjustment for Visual-Inertial SLAM
Technical University of Munich（慕尼黑工业大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；National University of Rosario（罗萨里奥国立大学）；CIFASIS (CONICET-UNR)（CIFASIS（CONICET-UNR））
The recent adoption of the Robot Operating System (ROS) as a software standard in robotics has contributed to novel solutions for several problems on the area. One such problem is known as Simultaneous Localization and Mapping (SLAM) with autonomous navigation, for which a number of algorithms from different classes are available as ROS packages ready to be used on any compatible robot. Many anticipated applications of autonomous mobile robots require for them to navigate in diverse complex environments without support from exterior infrastructures. To perform this on-board navigation, the rob…
A Map-free Deep Learning-based Framework for Gate-to-Gate Monocular Visual Navigation aboard Miniaturized Aerial Vehicles
Dalle Molle Institute for Artificial Intelligence (IDSIA)（达勒·莫勒人工智能研究所）；USI-SUPSI（瑞士意大利语区大学与瑞士应用科学大学联盟）；Integrated Systems Laboratory (IIS)（集成系统实验室）；ETHZ（苏黎世联邦理工学院）
