This paper details a system for fast visual exploration and search without prior map information. We leverage frontier based planning with both LiDAR and visual sensing and augment it with a perception module that contextually labels points in the surroundings from wide Field of View 2D LiDAR scans. The goal of the perception module is to recognize surrounding points more likely to be the search target in order to provide an informed prior on which to plan next best viewpoints. The robust map-free scan classifier used to label pixels in the robot's surroundings is trained from expert data collected using a simple cart platform equipped with a map-based classifier. We propose a novel utility function that accounts for the contextual data found from the classifier. The resulting viewpoints encourage the robot to explore points unlikely to be permanent in the environment, leading the robot to locate objects of interest faster than several existing baseline algorithms. Our proposed system is further validated in real-world search experiments for single and multiple search objects with a Spot robot in two unseen environments. Videos of experiments, implementation details and open source code can be found at https://sites.google.com/view/lives-2024/home.
Guangdong University of Technology（广东工业大学）；Southern University of Science and Technology（南方科技大学）
2D LiDAR SLAM (Simultaneous Localization and Mapping) is widely used in indoor environments due to its stability and flexibility. However, its mapping procedure is usually operated by a joystick in static environments, while indoor environments often are dynamic with moving objects such as people. The generated map with noisy points due to the dynamic objects is usually incomplete and distorted. To address this problem, we propose a framework of 2D-LiDAR-based SLAM without manual control that effectively excludes dynamic objects (people) and simplify the process for a robot to map an environme…
Integrating multiple LiDAR sensors can significantly enhance a robot's perception of the environment, enabling it to capture adequate measurements for simultaneous localization and mapping (SLAM). Indeed, solid-state LiDARs can bring in high resolution at a low cost to traditional spinning LiDARs in robotic applications. However, their reduced field of view (FoV) limits performance, particularly indoors. In this paper, we propose a tightly-coupled multi-modal multi-LiDAR-inertial SLAM system for surveying and mapping tasks. By taking advantage of both solid-state and spinnings LiDARs, and buil…
Faculty of Electrical Engineering, Czech Technical University in Prague（捷克技术大学电气工程学院）
Autonomous operation of UAVs in a closed environment requires precise and reliable pose estimate that can stabilize the UAV without using external localization systems such as GNSS. In this work, we are concerned with estimating the pose from laser scans generated by an inexpensive and lightweight LIDAR. We propose a localization system for lightweight (under 200g) LIDAR sensors with high reliability in arbitrary environments, where other methods fail. The general nature of the proposed method allows deployment in wide array of applications. Moreover, seamless transitioning between different k…
MDF: Multi-Modal Data Fusion with CNN-Based Object Detection for Enhanced Indoor Localization Using LiDAR-SLAM
Department of Artificial Intelligence Convergence, Pukyong National University（人工智能融合系，浦项国立大学）；School of Computer Science, University of Nottingham Ningbo China（计算机科学学院，诺丁汉国际大学）
Feasibility of Indoor Frame-Wise Lidar Semantic Segmentation via Distillation from Visual Foundation Model
1 Department of Earth Observation Science, Faculty of Geo-Information Science；Earth Observation (ITC), University of Twente, 7522 NB Enschede, The Netherlands
Applications involving autonomous navigation and planning of mobile agents can benefit greatly by employing online Simultaneous Localization and Mapping (SLAM) techniques, however, their proper implementation still warrants an efficient amalgamation with any offline path planning method that may be used for the particular application. In this paper, such a case of amalgamation is considered for a LiDAR-based indoor mapping system which presents itself as a 2D coverage path planning problem implemented along with online SLAM. This paper shows how classic offline Coverage Path Planning (CPP) can…
Robust Lifelong Indoor LiDAR Localization using the Area Graph
ShanghaiTech University（上海科技大学）；School of Information Science and Technology (SIST)（信息科学与技术学院）；Key Laboratory of Intelligent Perception and Human-Machine Collaboration（智能感知与人机协同重点实验室）
本文提出基于 Area Graph 的终身室内 3D LiDAR 定位方法，通过语义建筑特征、点云滤波及加权点到线 ICP 实现鲁棒全局定位与位姿跟踪。
Technische Hochschule Ingolstadt（英戈尔施塔特应用科学大学）；GeneSys Elektronik GmbH（吉内斯电子有限公司）；Universidad de Castilla-La Mancha（卡斯蒂利亚-拉曼恰大学）
The trend towards autonomous driving and the continuous research in the automotive area, like Advanced Driver Assistance Systems (ADAS), requires an accurate localization under all circumstances. An accurate estimation of the vehicle state is a basic requirement for any trajectory-planning algorithm. Still, even when the introduction of the GPS L5 band promises lane-accuracy, coverage limitations in roofed areas still have to be addressed. In this work, a method for high precision indoor positioning using a LiDAR is presented. The method is based on the combination of motion models with LiDAR …
Institute for Aerospace Studies (UTIAS), University of Toronto（多伦多大学航空航天研究所（UTIAS））；Apple（苹果公司）
We present a method for generating, predicting, and using Spatiotemporal Occupancy Grid Maps (SOGM), which embed future semantic information of real dynamic scenes. We present an auto-labeling process that creates SOGMs from noisy real navigation data. We use a 3D-2D feedforward architecture, trained to predict the future time steps of SOGMs, given 3D lidar frames as input. Our pipeline is entirely self-supervised, thus enabling lifelong learning for real robots. The network is composed of a 3D back-end that extracts rich features and enables the semantic segmentation of the lidar frames, and …
Pose estimation is a fundamental building block for robotic applications such as autonomous vehicles, UAV, and large scale augmented reality. It is also a prohibitive factor for those applications to be in mass production, since the state-of-the-art, centimeter-level pose estimation often requires long mapping procedures and expensive localization sensors, e.g. LiDAR and high precision GPS/IMU, etc. To overcome the cost barrier, we propose a neural network based solution to localize a consumer degree RGB camera within a prior sparse LiDAR map with comparable centimeter-level precision. We achi…
Zen Aerologiks；BMS College of Engineering（BMS工程学院）
The navigation system is at the heart of any mobile robot it comprises of SLAM and path planning units, which is utilized by the robot to generate a map of the environment, localize itself within it and determine an optimal a path to the destination. This paper describes the conceptualization, development, simulation and hardware implementation of GenNav a generic indoor navigation system for any mobile aerial or ground robot. The generalization is brought about by modularizing and creating independence between the software computation and hardware actuation units by providing an alternate sou…
Department of AI & Data Science, University of Hong Kong (HKU)（香港大学人工智能与数据科学系）；Department of Architecture and Civil Engineering, City University of Hong Kong (CityU)（香港城市大学建筑与土木工程系）；Humanoids and Human Centered Mechatronics Research Line, Italian Institute of Technology (IIT)（意大利技术研究院人形机器人与以人为本机电一体化研究组）
FGGS-LiDAR: Ultra-Fast, GPU-Accelerated Simulation from General 3DGS Models to LiDAR
Tsinghua University, Shenzhen, China（清华大学深圳研究院）；Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）；DISCOVER Robotics；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院（AIR））
Semantic2D: Enabling Semantic Scene Understanding with 2D Lidar Alone
School of Computing and Data Science, The University of Hong Kong（计算与数据科学学院，香港大学）；Department of Mechanical Engineering, Temple University（机械工程系， Temple大学）
University of Toronto Institute for Aerospace Studies（多伦多大学航空航天研究所）；Apple Inc.（苹果公司）
We present a self-supervised learning approach for the semantic segmentation of lidar frames. Our method is used to train a deep point cloud segmentation architecture without any human annotation. The annotation process is automated with the combination of simultaneous localization and mapping (SLAM) and ray-tracing algorithms. By performing multiple navigation sessions in the same environment, we are able to identify permanent structures, such as walls, and disentangle short-term and long-term movable objects, such as people and tables, respectively. New sessions can then be performed using a…
Singapore University of Technology and Design（新加坡科技设计大学）；Southwest University of Science and Technology（西南科技大学）
Autonomous robots operating in indoor and GPS denied environments can use LiDAR for SLAM instead. However, LiDARs do not perform well in geometrically-degraded environments, due to the challenge of loop closure detection and computational load to perform scan matching. Existing WiFi infrastructure can be exploited for localization and mapping with low hardware and computational cost. Yet, accurate pose estimation using WiFi is challenging as different signal values can be measured at the same location due to the unpredictability of signal propagation. Therefore, we introduce the use of WiFi fi…
Independently exploring unknown spaces or finding objects in an indoor environment is a daily but challenging task for visually impaired people. However, common 2D assistive systems lack depth relationships between various objects, resulting in difficulty to obtain accurate spatial layout and relative positions of objects. To tackle these issues, we propose HIDA, a lightweight assistive system based on 3D point cloud instance segmentation with a solid-state LiDAR sensor, for holistic indoor detection and avoidance. Our entire system consists of three hardware components, two interactive functi…
ShanghaiTech University（上海科技大学）；School of Information Science and Technology (SIST)（信息科学与技术学院）；Mobile Autonomous Robotic Systems Lab (MARS Lab)（移动自主机器人系统实验室）
Mobile robots depend on maps for localization, planning, and other applications. In indoor scenarios, there is often lots of clutter present, such as chairs, tables, other furniture, or plants. While mapping this clutter is important for certain applications, for example navigation, maps that represent just the immobile parts of the environment, i.e. walls, are needed for other applications, like room segmentation or long-term localization. In literature, approaches can be found that use a complete point cloud to remove the furniture in the room and generate a furniture free map. In contrast, …
Zhejiang University（浙江大学）；University of Delaware（特拉华大学）
This paper presents a tightly-coupled multi-sensor fusion algorithm termed LiDAR-inertial-camera fusion (LIC-Fusion), which efficiently fuses IMU measurements, sparse visual features, and extracted LiDAR points. In particular, the proposed LIC-Fusion performs online spatial and temporal sensor calibration between all three asynchronous sensors, in order to compensate for possible calibration variations. The key contribution is the optimal (up to linearization errors) multi-modal sensor fusion of detected and tracked sparse edge/surf feature points from LiDAR scans within an efficient MSCKF-bas…
In this paper, we evaluate eight popular and open-source 3D Lidar and visual SLAM (Simultaneous Localization and Mapping) algorithms, namely LOAM, Lego LOAM, LIO SAM, HDL Graph, ORB SLAM3, Basalt VIO, and SVO2. We have devised experiments both indoor and outdoor to investigate the effect of the following items: i) effect of mounting positions of the sensors, ii) effect of terrain type and vibration, iii) effect of motion (variation in linear and angular speed). We compare their performance in terms of relative and absolute pose error. We also provide comparison on their required computational …
Zhejiang University（浙江大学）；Hangzhou Dianzi University（杭州电子科技大学）；Meituan（美团）；University of Delaware（特拉华大学）
Currently, the improvement of LiDAR poses estimation accuracy is an urgent need for mobile robots. Research indicates that diverse LiDAR points have different influences on the accuracy of pose estimation. This study aimed to select a good point set to enhance accuracy. Accordingly, the sensitivity and uncertainty of LiDAR point residuals were formulated as a fundamental basis for derivation and analysis. High-sensitivity and low -uncertainty point residual terms are preferred to achieve higher pose estimation accuracy. The proposed selection method has been theoretically proven to be capable …
Saint Petersburg Electrotechnical University "LETI"（圣彼得堡国立电工大学“LETI”）
Modern laser SLAM (simultaneous localization and mapping) and structure from motion algorithms face the problem of processing redundant data. Even if a sensor does not move, it still continues to capture scans that should be processed. This paper presents the novel filter that allows dropping 2D scans that bring no new information to the system. Experiments on MIT and TUM datasets show that it is possible to drop more than half of the scans. Moreover thepaper describes the formulas that enable filter adaptation to a particular robot with known speed and characteristics of lidar. In addition, t…
EKF-Based Fusion of Wi-Fi/LiDAR/IMU for Indoor Localization and Navigation
School of Advanced Technology, Xi'an Jiaotong-Liverpool University（西安交通大学利物浦大学先进技术学院）；Department of Electrical Engineering and Electronics, University of Liverpool（利物浦大学电子工程系）；Zhejiang University of Technology（浙江工业大学）；School of Artificial Intelligence, Suzhou Vocational Institute of Industrial Technology（苏州工业职业技术学院人工智能学院）
Technical University of Munich（慕尼黑工业大学）；Sun Yat-sen University（中山大学）
Localization in the environment is an essential navigational capability for animals and mobile robots. In the indoor environment, the global localization problem remains challenging to be perfectly solved with probabilistic methods. However, animals are able to instinctively localize themselves with much less effort. Therefore, it is intriguing and promising to seek biological inspiration from animals. In this paper, we present a biologically-inspired global localization system using a LiDAR sensor that utilizes a hippocampal model and a landmark-based re-localization approach. The experiment …
Robust Dynamic Object Detection in Cluttered Indoor Scenes via Learned Spatiotemporal Cues
Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente（代尔夫特理工大学电子工程、数学与计算机科学学院）；Department of Electrical Engineering, Wright State University（怀特塞德大学电子工程系）；Massachusetts Institute of Technology（麻省理工学院）
This paper presents a computationally efficient and robust LiDAR-inertial odometry framework. We fuse LiDAR feature points with IMU data using a tightly-coupled iterated extended Kalman filter to allow robust navigation in fast-motion, noisy or cluttered environments where degeneration occurs. To lower the computation load in the presence of large number of measurements, we present a new formula to compute the Kalman gain. The new formula has computation load depending on the state dimension instead of the measurement dimension. The proposed method and its implementation are tested in various …
RoboLoc: A Benchmark Dataset for Point Place Recognition and Localization in Indoor-Outdoor Integrated Environments
Department of Data, Network and AI, Ajou University（数据、网络与人工智能系，亚运会大学）；Korea Railroad Research Institute（韩国铁路研究院）；Department of Mobility Engineering, Ajou University（移动工程系，亚运会大学）
The University of Hong Kong（香港大学）；Southern University of Science and Technology（南方科技大学）
In this letter, we present a novel method for automatic extrinsic calibration of high-resolution LiDARs and RGB cameras in targetless environments. Our approach does not require checkerboards but can achieve pixel-level accuracy by aligning natural edge features in the two sensors. On the theory level, we analyze the constraints imposed by edge features and the sensitivity of calibration accuracy with respect to edge distribution in the scene. On the implementation level, we carefully investigate the physical measuring principles of LiDARs and propose an efficient and accurate LiDAR edge extra…
ETH Zürich（苏黎世联邦理工学院）；Max Planck ETH Center for Learning Systems（马克斯·普朗克-苏黎世联邦理工学院学习系统中心）
Reliable robot pose estimation is a key building block of many robot autonomy pipelines, with LiDAR localization being an active research domain. In this work, a versatile self-supervised LiDAR odometry estimation method is presented, in order to enable the efficient utilization of all available LiDAR data while maintaining real-time performance. The proposed approach selectively applies geometric losses during training, being cognizant of the amount of information that can be extracted from scan points. In addition, no labeled or ground-truth data is required, hence making the presented appro…
Shanghai Jiao Tong University（上海交通大学）；Institute of Medical Robotics（医疗机器人研究院）
Long-term scene changes present challenges to localization systems using a pre-built map. This paper presents a LiDAR-based system that can provide robust localization against those challenges. Our method starts with activation of a mapping process temporarily when global matching towards the pre-built map is unreliable. The temporary map will be merged onto the pre-built map for later localization runs once reliable matching is obtained again. We further integrate a LiDAR inertial odometry (LIO) to provide motion-compensated LiDAR scans and a reliable initial pose guess for the global matchin…
This paper addresses the issues of unmanned aerial vehicle (UAV) indoor navigation, specifically in areas where GPS and magnetometer sensor measurements are unavailable or unreliable. The proposed solution is to use an error state extended Kalman filter (ES -EKF) in the context of multi-sensor fusion. Its implementation is adapted to fuse measurements from multiple sensor sources and the state model is extended to account for sensor drift and possible calibration inaccuracies. Experimental validation is performed by fusing IMU data obtained from the PixHawk 2.1 flight controller with pose meas…
In this letter, we propose a robust, real-time tightly-coupled multi-sensor fusion framework, which fuses measurement from LiDAR, inertial sensor, and visual camera to achieve robust and accurate state estimation. Our proposed framework is composed of two parts: the filter-based odometry and factor graph optimization. To guarantee real-time performance, we estimate the state within the framework of error-state iterated Kalman-filter, and further improve the overall precision with our factor graph optimization. Taking advantage of measurement from all individual sensors, our algorithm is robust…
Localization of a wireless mobile device or a robot in indoor and GPS-denied environments is a difficult problem, particularly in dynamic scenarios where traditional cameras and LIDAR-based alternative sensing and localization modalities may fail. We propose a method for estimating the location of a mobile robot in relation to static wireless sensor nodes (WSN) deployed in the environment. The method employs a novel particle filter that updates its weights using a Gauss probability over Direction of Arrival (DOA) estimate in conjunction with the mobile robot's mobility model. We evaluate and v…
Increasing the density of the 3D LiDAR point cloud is appealing for many applications in robotics. However, high-density LiDAR sensors are usually costly and still limited to a level of coverage per scan (e.g., 128 channels). Meanwhile, denser point cloud scans and maps mean larger volumes to store and longer times to transmit. Existing works focus on either improving point cloud density or compressing its size. This paper aims to design a novel 3D point cloud representation that can continuously increase point cloud density while reducing its storage and transmitting size. The pipeline of the…
KIOS Center of Excellence, University of Cyprus（塞浦路斯大学KIOS卓越中心）；University of Cyprus（塞浦路斯大学）
Indoor exploration is an important task in disaster relief, emergency response scenarios, and Search And Rescue (SAR) missions. Unmanned Aerial Vehicle (UAV) systems can aid first responders by maneuvering autonomously in areas inside buildings dangerous for humans to traverse, exploring the interior, and providing an accurate and reliable indoor map before the emergency response team takes action. Due to the challenging conditions in such scenarios and the inherent battery limitations and time constraints, we investigate 2D autonomous exploration strategies (e.g., based on 2D LiDAR) for mappi…
