We propose a novel online learning algorithm, called SpCoSLAM 2.0, for spatial concepts and lexical acquisition with high accuracy and scalability. Previously, we proposed SpCoSLAM as an online learning algorithm based on unsupervised Bayesian probabilistic model that integrates multimodal place categorization, lexical acquisition, and SLAM. However, our original algorithm had limited estimation accuracy owing to the influence of the early stages of learning, and increased computational complexity with added training data. Therefore, we introduce techniques such as fixed-lag rejuvenation to reduce the calculation time while maintaining an accuracy higher than that of the original algorithm. The results show that, in terms of estimation accuracy, the proposed algorithm exceeds the original algorithm and is comparable to batch learning. In addition, the calculation time of the proposed algorithm does not depend on the amount of training data and becomes constant for each step of the scalable algorithm. Our approach will contribute to the realization of long-term spatial language interactions between humans and robots.
Ritsumeikan University（立命馆大学）；National Institute of Informatics（国立信息学研究所）；SOKENDAI (The Graduate University for Advanced Studies)（综合研究大学院大学）
In this paper, we propose an online learning algorithm based on a Rao-Blackwellized particle filter for spatial concept acquisition and mapping. We have proposed a nonparametric Bayesian spatial concept acquisition model (SpCoA). We propose a novel method (SpCoSLAM) integrating SpCoA and FastSLAM in the theoretical framework of the Bayesian generative model. The proposed method can simultaneously learn place categories and lexicons while incrementally generating an environmental map. Furthermore, the proposed method has scene image features and a language model added to SpCoA. In the experimen…
Autonomous underwater vehicles (AUVs) are becoming standard tools for underwater exploration and seabed mapping in both scientific and industrial applications \cite{graham2022rapid, stenius2022system}. Their capacity to dive untethered allows them to reach areas inaccessible to surface vessels and to collect data more closely to the seafloor, regardless of the water depth. However, their navigation autonomy remains bounded by the accuracy of their dead reckoning (DR) estimate of their global position, severely limited in the absence of a priori maps of the area and GPS signal. Global localizat…
Modern 3D laser-range scanners have a high data rate, making online simultaneous localization and mapping (SLAM) computationally challenging. Recursive state estimation techniques are efficient but commit to a state estimate immediately after a new scan is made, which may lead to misalignments of measurements. We present a 3D SLAM approach that allows for refining alignments during online mapping. Our method is based on efficient local mapping and a hierarchical optimization back-end. Measurements of a 3D laser scanner are aggregated in local multiresolution maps by means of surfel-based regis…
Hong Kong University of Science and Technology（香港科技大学）
Combining multiple LiDARs enables a robot to maximize its perceptual awareness of environments and obtain sufficient measurements, which is promising for simultaneous localization and mapping (SLAM). This paper proposes a system to achieve robust and simultaneous extrinsic calibration, odometry, and mapping for multiple LiDARs. Our approach starts with measurement preprocessing to extract edge and planar features from raw measurements. After a motion and extrinsic initialization procedure, a sliding window-based multi-LiDAR odometry runs onboard to estimate poses with online calibration refine…
This paper develops a Bayesian continuous 3D semantic occupancy map from noisy point clouds by generalizing the Bayesian kernel inference model for building occupancy maps, a binary problem, to semantic maps, a multi-class problem. The proposed method provides a unified probabilistic model for both occupancy and semantic probabilities and nicely reverts to the original occupancy mapping framework when only one occupied class exists in obtained measurements. The Bayesian spatial kernel inference relaxes the independent grid assumption and brings smoothness and continuity to the map inference, e…
The University of Hong Kong（香港大学）；Southern University of Science and Technology（南方科技大学）
This paper proposes an efficient and probabilistic adaptive voxel mapping method for LiDAR odometry. The map is a collection of voxels; each contains one plane (or edge) feature that enables the probabilistic representation of the environment and accurate registration of a new LiDAR scan. We further analyze the need for coarse-to-fine voxel mapping and then use a novel voxel map organized by a Hash table and octrees to build and update the map efficiently. We apply the proposed voxel map to an iterated extended Kalman filter and construct a maximum a posteriori probability problem for pose est…
Applications involving autonomous navigation and planning of mobile agents can benefit greatly by employing online Simultaneous Localization and Mapping (SLAM) techniques, however, their proper implementation still warrants an efficient amalgamation with any offline path planning method that may be used for the particular application. In this paper, such a case of amalgamation is considered for a LiDAR-based indoor mapping system which presents itself as a 2D coverage path planning problem implemented along with online SLAM. This paper shows how classic offline Coverage Path Planning (CPP) can…
CSMapping: Scalable Crowdsourced Semantic Mapping and Topology Inference for Autonomous Driving
the Department of Electronic and Computer Engineering, the Hong Kong University of Science and Technology, Hong Kong, China（电子与计算机工程系，香港科技大学，香港，中国）；the Academy for Engineering and Technology, Fudan University, Shanghai, China（工程与技术学院，复旦大学，上海，中国）；Zhuoyu Technology Co., Ltd., Shenzhen, China（珠海优创科技有限公司，深圳，中国）
Otto von Guericke University Magdeburg（马格德堡奥托·冯·格里克大学）
We present a real-time semantic mapping approach for mobile vision systems with a 2D to 3D object detection pipeline and rapid data association for generated landmarks. Besides the semantic map enrichment the associated detections are further introduced as semantic constraints into a simultaneous localization and mapping (SLAM) system for pose correction purposes. This way, we are able generate additional meaningful information that allows to achieve higher-level tasks, while simultaneously leveraging the view-invariance of object detections to improve the accuracy and the robustness of the od…
The Hong Kong University of Science and Technology（香港科技大学）
In this paper, we propose a novel dense surfel mapping system that scales well in different environments with only CPU computation. Using a sparse SLAM system to estimate camera poses, the proposed mapping system can fuse intensity images and depth images into a globally consistent model. The system is carefully designed so that it can build from room-scale environments to urban-scale environments using depth images from RGB-D cameras, stereo cameras or even a monocular camera. First, superpixels extracted from both intensity and depth images are used to model surfels in the system. superpixel…
We propose PanopticFusion, a novel online volumetric semantic mapping system at the level of stuff and things. In contrast to previous semantic mapping systems, PanopticFusion is able to densely predict class labels of a background region (stuff) and individually segment arbitrary foreground objects (things). In addition, our system has the capability to reconstruct a large-scale scene and extract a labeled mesh thanks to its use of a spatially hashed volumetric map representation. Our system first predicts pixel-wise panoptic labels (class labels for stuff regions and instance IDs for thing r…
R-VoxelMap: Accurate Voxel Mapping with Recursive Plane Fitting for Online LiDAR Odometry
College of Artificial Intelligence, Institute of Robotics and Automatic Information System, and the Tianjin Key Laboratory of Intelligent Robotics（人工智能学院、机器人与自动化信息系统研究所以及天津智能机器人重点实验室）
University of California, San Diego（加利福尼亚大学圣迭戈分校）；U.S. Army Research Laboratory（美国陆军研究实验室）
We develop an online probabilistic metric-semantic mapping approach for mobile robot teams relying on streaming RGB-D observations. The generated maps contain full continuous distributional information about the geometric surfaces and semantic labels (e.g., chair, table, wall). Our approach is based on online Gaussian Process (GP) training and inference, and avoids the complexity of GP classification by regressing a truncated signed distance function (TSDF) of the regions occupied by different semantic classes. Online regression is enabled through a sparse pseudo-point approximation of the GP …
This paper is a technical report about our submission for the ECCV 2018 3DRMS Workshop Challenge on Semantic 3D Reconstruction \cite{Tylecek2018rms}. In this paper, we address 3D semantic reconstruction for autonomous navigation using co-learning of depth map and semantic segmentation. The core of our pipeline is a deep multi-task neural network which tightly refines depth and also produces accurate semantic segmentation maps. Its inputs are an image and a raw depth map produced from a pair of images by standard stereo vision. The resulting semantic 3D point clouds are then merged in order to …
We present a fast, scalable, and accurate Simultaneous Localization and Mapping (SLAM) system that represents indoor scenes as a graph of objects. Leveraging the observation that artificial environments are structured and occupied by recognizable objects, we show that a compositional scalable object mapping formulation is amenable to a robust SLAM solution for drift-free large scale indoor reconstruction. To achieve this, we propose a novel semantically assisted data association strategy that obtains unambiguous persistent object landmarks, and a 2.5D compositional rendering method that enable…
Adaptive World Memory 3D Foundation Model for Scalable 3D Mapping, Localization, and Rendering
Shanghai Jiao Tong University（上海交通大学）；State Key Laboratory of Avionics Integration and Aviation System-of-Systems Synthesis（航空电子综合与航空系统体系综合国家重点实验室）；Shanghai Key Laboratory of Navigation and Location Based Services（上海市导航与位置服务重点实验室）；Nanyang Technological University（南洋理工大学）；University of Technology Nuremberg（纽伦堡工业大学）
OpenNavMap: Multi-Session Appearance-Based Topometric Mapping for Scalable Visual Navigation
Robot Perception and Learning Lab, Intelligent Robotics, Department of Computer Science, University College London（机器人感知与学习实验室，智能机器人，计算机科学系，伦敦大学学院）；Department of Computer Science and Engineering, Hong Kong University of Science and Technology（计算机科学与工程系，香港科学与技术大学）；Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology（电子与计算机工程系，香港科学与技术大学）；Institute of Robotics and Automatic Information System, Nankai University（机器人与自动信息系统研究所，南开大学）；College of control science and engineering, Zhejiang University（控制科学与工程学院，浙江大学）；AI Centre, Department of Computer Science, University College London（人工智能中心，计算机科学系，伦敦大学学院）
Resource-Efficient Cooperative Online Scalar Field Mapping via Distributed Sparse Gaussian Process Regression
Zhejiang University（浙江大学）；Jinhua Institute of Zhejiang University（浙江大学金华研究院）；National Key Laboratory of Industrial Control Technology, Zhejiang University（浙江大学工业控制技术国家重点实验室）
University of Cambridge（剑桥大学）；DEVCOM Army Research Laboratory（美国陆军作战能力发展司令部陆军研究实验室）
Many multi-robot planning problems are burdened by the curse of dimensionality, which compounds the difficulty of applying solutions to large-scale problem instances. The use of learning-based methods in multi-robot planning holds great promise as it enables us to offload the online computational burden of expensive, yet optimal solvers, to an offline learning procedure. Simply put, the idea is to train a policy to copy an optimal pattern generated by a small-scale system, and then transfer that policy to much larger systems, in the hope that the learned strategy scales, while maintaining near…
CIRS Underwater Vision and Robotics Research Centre（CIRS水下视觉与机器人研究中心）；Universitat de Girona（赫罗纳大学）；VICOROB VIsió per COmputador i ROBòtica（VICOROB计算机视觉与机器人实验室）
Safe autonomous navigation is an essential and challenging problem for robots operating in highly unstructured or completely unknown environments. Under these conditions, not only robotic systems must deal with limited localisation information, but also their manoeuvrability is constrained by their dynamics and often suffer from uncertainty. In order to cope with these constraints, this manuscript proposes an uncertainty-based framework for mapping and planning feasible motions online with probabilistic safety-guarantees. The proposed approach deals with the motion, probabilistic safety, and o…
Traditional dense volumetric representations for robotic mapping make simplifying assumptions about sensor noise characteristics due to computational constraints. We present a framework that, unlike conventional occupancy grid maps, explicitly models the sensor ray formation for a depth sensor via a Markov Random Field and performs loopy belief propagation to infer the marginal probability of occupancy at each voxel in a map. By explicitly reasoning about occlusions our approach models the correlations between adjacent voxels in the map. Further, by incorporating learnt sensor noise characteri…
Online Monocular Lane Mapping Using Catmull-Rom Spline
The Hong Kong University of Science and Technology（香港科技大学）；HKUST-DJI Joint Innovation Laboratory（香港科技大学-大疆创新联合创新实验室）；Hong Kong Center for Construction Robotics（香港建造机器人中心）
In many applications, maintaining a consistent dense map of the environment is key to enabling robotic platforms to perform higher level decision making. Several works have addressed the challenge of creating precise dense 3D maps from visual sensors providing depth information. However, during operation over longer missions, reconstructions can easily become inconsistent due to accumulated camera tracking error and delayed loop closure. Without explicitly addressing the problem of map consistency, recovery from such distortions tends to be difficult. We present a novel system for dense 3D map…
MIT-WHOI Joint Program in Oceanography/Applied Ocean Science and Engineering（麻省理工学院-伍兹霍尔海洋研究所联合海洋学/应用海洋科学与工程项目）；Massachusetts Institute of Technology (MIT)（麻省理工学院）；Woods Hole Oceanographic Institution (WHOI)（伍兹霍尔海洋研究所）
We present a solution to multi-robot distributed semantic mapping of novel and unfamiliar environments. Most state-of-the-art semantic mapping systems are based on supervised learning algorithms that cannot classify novel observations online. While unsupervised learning algorithms can invent labels for novel observations, approaches to detect when multiple robots have independently developed their own labels for the same new class are prone to erroneous or inconsistent matches. These issues worsen as the number of robots in the system increases and prevent fusing the local maps produced by eac…
The University of Sydney（悉尼大学）；Stanford University（斯坦福大学）；NVIDIA Research（英伟达研究院）
Creating accurate spatial representations that take into account uncertainty is critical for autonomous robots to safely navigate in unstructured environments. Although recent LIDAR based mapping techniques can produce robust occupancy maps, learning the parameters of such models demand considerable computational time, discouraging them from being used in real-time and large-scale applications such as autonomous driving. Recognizing the fact that real-world structures exhibit similar geometric features across a variety of urban environments, in this paper, we argue that it is redundant to lear…
For robotic interaction in environments shared with other agents, access to volumetric and semantic maps of the scene is crucial. However, such environments are inevitably subject to long-term changes, which the map needs to account for. We thus propose panoptic multi-TSDFs as a novel representation for multi-resolution volumetric mapping in changing environments. By leveraging high-level information for 3D reconstruction, our proposed system allocates high resolution only where needed. Through reasoning on the object level, semantic consistency over time is achieved. This enables our method t…
University of Toronto（多伦多大学）；PES Institute of Technology（PES理工学院）
Real-time 3D mapping is a critical component in many important applications today including robotics, AR/VR, and 3D visualization. 3D mapping involves continuously fusing depth maps obtained from depth sensors in phones, robots, and autonomous vehicles into a single 3D representative model of the scene. Many important applications, e.g., global path planning and trajectory generation in micro aerial vehicles, require the construction of large maps at high resolutions. In this work, we identify mapping, i.e., construction and updates of 3D maps to be a critical bottleneck in these applications.…
FERMI: Flexible Radio Mapping with a Hybrid Propagation Model and Scalable Autonomous Data Collection
Southern University of Science and Technology（南方科技大学）；The University of Hong Kong（香港大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科学理工大学（广州））；Sun Yat-Sen University（中山大学）
