We propose a lifelong 3D mapping framework that is modular, cloud-native by design and more importantly, works for both hand-held and robot-mounted 3D LiDAR mapping systems. Our proposed framework comprises of dynamic point removal, multi-session map alignment, map change detection and map version control. First, our sensor-setup agnostic dynamic point removal algorithm works seamlessly with both hand-held and robot-mounted setups to produce clean static 3D maps. Second, the multi-session map alignment aligns these clean static maps automatically, without manual parameter fine-tuning, into a single reference frame, using a two stage approach based on feature descriptor matching and fine registration. Third, our novel map change detection identifies positive and negative changes between two aligned maps. Finally, the map version control maintains a single base map that represents the current state of the environment, and stores the detected positive and negative changes, and boundary information. Our unique map version control system can reconstruct any of the previous clean session maps and allows users to query changes between any two random mapping sessions, all without storing any input raw session maps, making it very unique. Extensive experiments are performed using hand-held commercial LiDAR mapping devices and open-source robot-mounted LiDAR SLAM algorithms to evaluate each module and the whole 3D lifelong mapping framework.
Long-term 3D map management is a fundamental capability required by a robot to reliably navigate in the non-stationary real-world. This paper develops open-source, modular, and readily available LiDAR-based lifelong mapping for urban sites. This is achieved by dividing the problem into successive subproblems: multi-session SLAM (MSS), high/low dynamic change detection, and positive/negative change management. The proposed method leverages MSS and handles potential trajectory error; thus, good initial alignment is not required for change detection. Our change management scheme preserves efficac…
Carnegie Mellon University（卡内基梅隆大学）；University of California San Diego（加州大学圣地亚哥分校）
We present BioSLAM, a lifelong SLAM framework for learning various new appearances incrementally and maintaining accurate place recognition for previously visited areas. Unlike humans, artificial neural networks suffer from catastrophic forgetting and may forget the previously visited areas when trained with new arrivals. For humans, researchers discover that there exists a memory replay mechanism in the brain to keep the neuron active for previous events. Inspired by this discovery, BioSLAM designs a gated generative replay to control the robot's learning behavior based on the feedback reward…
In this paper, we describe a strategy for training neural networks for object detection in range images obtained from one type of LiDAR sensor using labeled data from a different type of LiDAR sensor. Additionally, an efficient model for object detection in range images for use in self-driving cars is presented. Currently, the highest performing algorithms for object detection from LiDAR measurements are based on neural networks. Training these networks using supervised learning requires large annotated datasets. Therefore, most research using neural networks for object detection from LiDAR po…
Oxford Robotics Institute（牛津机器人研究所）；University of Oxford（牛津大学）
In this paper we present a large dataset with a variety of mobile mapping sensors collected using a handheld device carried at typical walking speeds for nearly 2.2 km through New College, Oxford. The dataset includes data from two commercially available devices - a stereoscopic-inertial camera and a multi-beam 3D LiDAR, which also provides inertial measurements. Additionally, we used a tripod-mounted survey grade LiDAR scanner to capture a detailed millimeter-accurate 3D map of the test location (containing $\sim$290 million points). Using the map we inferred centimeter-accurate 6 Degree of F…
LiLoc: Lifelong Localization using Adaptive Submap Joining and Egocentric Factor Graph
Zhejiang University（浙江大学）；Southeast University（东南大学）；National University of Singapore（新加坡国立大学）；Technical University of Munich（慕尼黑工业大学）；Google Zurich（谷歌苏黎世分公司）
Lidar-based simultaneous localization and mapping (SLAM) approaches have obtained considerable success in autonomous robotic systems. This is in part owing to the high-accuracy of robust SLAM algorithms and the emergence of new and lower-cost lidar products. This study benchmarks current state-of-the-art lidar SLAM algorithms with a multi-modal lidar sensor setup showcasing diverse scanning modalities (spinning and solid-state) and sensing technologies, and lidar cameras, mounted on a mobile sensing and computing platform. We extend our previous multi-modal multi-lidar dataset with additional …
Cross-view Domain Generalization via Geometric Consistency for LiDAR Semantic Segmentation
School of Aeronautic Engineering, Changsha University of Science and Technology（长沙理工大学航空工程学院）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院航空信息研究所）；School of Automation, Central South University（中南大学自动化学院）；University of British Columbia（不列颠哥伦比亚大学）
Learning Vision-Based Omnidirectional Navigation: A Teacher-Student Approach Using Monocular Depth Estimation
Fraunhofer Institute for Material Flow and Logistics（弗劳恩霍夫材料流与物流研究所）；The Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔人工智能与机器学习研究所）
Robust Lifelong Indoor LiDAR Localization using the Area Graph
ShanghaiTech University（上海科技大学）；School of Information Science and Technology (SIST)（信息科学与技术学院）；Key Laboratory of Intelligent Perception and Human-Machine Collaboration（智能感知与人机协同重点实验室）
本文提出基于 Area Graph 的终身室内 3D LiDAR 定位方法，通过语义建筑特征、点云滤波及加权点到线 ICP 实现鲁棒全局定位与位姿跟踪。
Intel’s Internet of Things Group（英特尔物联网事业部）；Nagoya University（名古屋大学）；The University of Tokyo（东京大学）
This paper describes an optimized single-stage deep convolutional neural network to detect objects in urban environments, using nothing more than point cloud data. This feature enables our method to work regardless the time of the day and the lighting conditions.The proposed network structure employs dilated convolutions to gradually increase the perceptive field as depth increases, this helps to reduce the computation time by about 30%. The network input consists of five perspective representations of the unorganized point cloud data. The network outputs an objectness map and the bounding box…
University of Toronto Institute for Aerospace Studies (UTIAS)（多伦多大学航空航天研究所）；ETH Zürich（苏黎世联邦理工学院）；Apple Inc.（苹果公司）
We present a novel method for generating, predicting, and using Spatiotemporal Occupancy Grid Maps (SOGM), which embed future information of dynamic scenes. Our automated generation process creates groundtruth SOGMs from previous navigation data. We build on prior work to annotate lidar points based on their dynamic properties, which are then projected on time-stamped 2D grids: SOGMs. We design a 3D-2D feedforward architecture, trained to predict the future time steps of SOGMs, given 3D lidar frames as input. Our pipeline is entirely self-supervised, thus enabling lifelong learning for robots.…
Energy-Efficient SLAM via Joint Design of Sensing, Communication, and Exploration Speed
Southern University of Science and Technology（南方科技大学）；Harbin Institute of Technoglogy (Shenzhen)（哈尔滨工业大学（深圳））；Shenzhen Research Institute of Big Data（深圳大数据研究院）；The School of Electronics and Communication Engineering（电子与通信工程学院）；Sun Yat-sen University（中山大学）
RS2AD: End-to-End Autonomous Driving Data Generation from Roadside Sensor Observations
School of Vehicle and Mobility, Tsinghua University（清华大学车辆与移动系统学院）；State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University（清华大学智能绿色车辆与移动系统国家重点实验室）；School of Instrumentation and Optoelectronic Engineering, BeiHang University（北航仪器与光电工程学院）；Department of Automation, Tsinghua University（清华大学自动化系）
Xiamen University（厦门大学）；ShanghaiTech University（上海科技大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
We present SLOPER4D, a novel scene-aware dataset collected in large urban environments to facilitate the research of global human pose estimation (GHPE) with human-scene interaction in the wild. Employing a head-mounted device integrated with a LiDAR and camera, we record 12 human subjects' activities over 10 diverse urban scenes from an egocentric view. Frame-wise annotations for 2D key points, 3D pose parameters, and global translations are provided, together with reconstructed scene point clouds. To obtain accurate 3D ground truth in such large dynamic scenes, we propose a joint optimizatio…
We propose a novel, canopy density estimation solution using a 3D ray cloud representation for perennial horticultural crops at the field scale. To attain high spatial and temporal fidelity in field conditions, we propose the application of continuous-time 3D SLAM (Simultaneous Localisation and Mapping) to a spinning lidar payload (AgScan3D) mounted on a moving farm vehicle. The AgScan3D data is processed through a Continuous-Time SLAM algorithm into a globally registered 3D ray cloud. The global ray cloud is a canonical data format (a digital twin) from which we can compare vineyard snapshots…
SMapper: A Multi-Modal Data Acquisition Platform for SLAM Benchmarking
Automation and Robotics Research Group (ARG), Interdisciplinary Centre for Security, Reliability, and Trust (SnT), University of Luxembourg（卢森堡大学自动化与机器人研究组（ARG）、安全、可靠与信任跨学科研究中心（SnT））
This work describes the automatic registration of a large network (approximately 40) of fixed, ceiling-mounted environment cameras spread over a large area (approximately 800 squared meters) using a mobile calibration robot equipped with a single upward-facing fisheye camera and a backlit ArUco marker for easy detection. The fisheye camera is used to do visual odometry (VO), and the ArUco marker facilitates easy detection of the calibration robot in the environment cameras. In addition, the fisheye camera is also able to detect the environment cameras. This two-way, bidirectional detection con…
Unmanned aerial vehicles (UAVs) have been used for many applications in recent years, from urban search and rescue, to agricultural surveying, to autonomous underground mine exploration. However, deploying UAVs in tight, indoor spaces, especially close to humans, remains a challenge. One solution, when limited payload is required, is to use micro-UAVs, which pose less risk to humans and typically cost less to replace after a crash. However, micro-UAVs can only carry a limited sensor suite, e.g. a monocular camera instead of a stereo pair or LiDAR, complicating tasks like dense mapping and mark…
The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong Polytechnic University（香港理工大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute（香港科技大学深港协同创新研究院）
With the fast development of autonomous driving technologies, there is an increasing demand for high-definition (HD) maps, which provide reliable and robust prior information about the static part of the traffic environments. As one of the important elements in HD maps, road lane centerline is critical for downstream tasks, such as prediction and planning. Manually annotating centerlines for road lanes in HD maps is labor-intensive, expensive and inefficient, severely restricting the wide applications of autonomous driving systems. Previous work seldom explores the lane centerline detection pr…
Efficient LiDAR Bundle Adjustment for Multi-Scan Alignment Utilizing Continuous-Time Trajectories
Center for Robotics, University of Bonn（波恩大学机器人中心）；University of Oxford（牛津大学）；Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔机器学习与人工智能研究所）
Institute for Aerospace Studies (UTIAS), University of Toronto（多伦多大学航空航天研究所（UTIAS））；Apple（苹果公司）
We present a method for generating, predicting, and using Spatiotemporal Occupancy Grid Maps (SOGM), which embed future semantic information of real dynamic scenes. We present an auto-labeling process that creates SOGMs from noisy real navigation data. We use a 3D-2D feedforward architecture, trained to predict the future time steps of SOGMs, given 3D lidar frames as input. Our pipeline is entirely self-supervised, thus enabling lifelong learning for real robots. The network is composed of a 3D back-end that extracts rich features and enables the semantic segmentation of the lidar frames, and …
PERAL: Perception-Aware Motion Control for Passive LiDAR Excitation in Spherical Robots
Centre for Advanced Robotics Technology Innovation (CARTIN), School of Electrical and Electronic Engineering, Nanyang Technological University（先进机器人技术创新中心（CARTIN）、电气与电子工程学院、南洋理工大学）
LiMo-Calib: On-Site Fast LiDAR-Motor Calibration for Quadruped Robot-Based Panoramic 3D Sensing System
China-Singapore International Joint Research Institute (CSIJRI)（中新国际联合研究院）；School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）
Large-Scale Tunnel Air-Ground Collaboration With FLISP: Fast LiDAR-IMU Synchronized Path Planner
Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University（同济大学上海自主智能无人系统科学中心）；State Key Laboratory of Autonomous Intelligent Unmanned Systems, Tongji University（同济大学自主智能无人系统全国重点实验室）；College of Electronic and Information Engineering, Tongji University（同济大学电子与信息工程学院）
Biomedical Research Institute of Malaga (IBIMA)（马拉加生物医学研究所（IBIMA））；University of Malaga（马拉加大学）
In order to fuse measurements from multiple sensors mounted on a mobile robot, it is needed to express them in a common reference system through their relative spatial transformations. In this paper, we present a method to estimate the full 6DoF extrinsic calibration parameters of multiple heterogeneous sensors (Lidars, Depth and RGB cameras) suitable for automatic execution on a mobile robot. Our method computes the 2D calibration parameters (x, y, yaw) through a motion-based approach, while for the remaining 3 parameters (z, pitch, roll) it requires the observation of the ground plane for a …
