LIV-GaussMap: LiDAR-Inertial-Visual Fusion for Real-time 3D Radiance Field Map Rendering
The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The University of Hong Kong（香港大学）
We introduce an integrated precise LiDAR, Inertial, and Visual (LIV) multimodal sensor fused mapping system that builds on the differentiable \pre{surface splatting }\now{Gaussians} to improve the mapping fidelity, quality, and structural accuracy. Notably, this is also a novel form of tightly coupled map for LiDAR-visual-inertial sensor fusion.
This system leverages the complementary characteristics of LiDAR and visual data to capture the geometric structures of large-scale 3D scenes and restore their visual surface information with high fidelity. The initialization for the scene's surface Gaussians and the sensor's poses of each frame are obtained using a LiDAR-inertial system with the feature of size-adaptive voxels. Then, we optimized and refined the Gaussians using visual-derived photometric gradients to optimize their quality and density.
Our method is compatible with various types of LiDAR, including solid-state and mechanical LiDAR, supporting both repetitive and non-repetitive scanning modes. Bolstering structure construction through LiDAR and facilitating real-time generation of photorealistic renderings across diverse LIV datasets. It showcases notable resilience and versatility in generating real-time photorealistic scenes potentially for digital twins and virtual reality, while also holding potential applicability in real-time SLAM and robotics domains.
We release our software and hardware and self-collected datasets to benefit the community.
Simultaneous localization and mapping (SLAM) are crucial for autonomous robots (e.g., self-driving cars, autonomous drones), 3D mapping systems, and AR/VR applications. This work proposed a novel LiDAR-inertial-visual fusion framework termed R$^3$LIVE++ to achieve robust and accurate state estimation while simultaneously reconstructing the radiance map on the fly. R$^3$LIVE++ consists of a LiDAR-inertial odometry (LIO) and a visual-inertial odometry (VIO), both running in real-time. The LIO subsystem utilizes the measurements from a LiDAR for reconstructing the geometric structure (i.e., the p…
Omni-LIVO: Robust RGB-Colored Multi-Camera Visual-Inertial-LiDAR Odometry via Photometric Migration and ESIKF Fusion
Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences（中国科学院大学杭州高等研究院）；Advanced Laser Technology Lab of Anhui Province（安徽省先进激光技术实验室）；Key Laboratory of Space Active Opto-Electronics Technology, Shanghai Institute of Technical Physics, Chinese Academy of Sciences（中国科学院上海技术物理研究所空间主动光电技术重点实验室）
Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）
We present a novel tightly-coupled LiDAR-inertial odometry and mapping scheme for both solid-state and mechanical LiDARs. As frontend, a feature-based lightweight LiDAR odometry provides fast motion estimates for adaptive keyframe selection. As backend, a hierarchical keyframe-based sliding window optimization is performed through marginalization for directly fusing IMU and LiDAR measurements. For the Livox Horizon, a newly released solid-state LiDAR, a novel feature extraction method is proposed to handle its irregular scan pattern during preprocessing. LiLi-OM (Livox LiDAR-inertial odometry …
School of System Design and Intelligent Manufacturing, Southern University of Science and Technology（南方科技大学系统设计与智能制造学院）
Combining lidar in camera-based simultaneous localization and mapping (SLAM) is an effective method in improving overall accuracy, especially at a large scale outdoor scenario. Recent development of low-cost lidars (e.g. Livox lidar) enable us to explore such SLAM systems with lower budget and higher performance. In this paper we propose CamVox by adapting Livox lidars into visual SLAM (ORB-SLAM2) by exploring the lidars' unique features. Based on the non-repeating nature of Livox lidars, we propose an automatic lidar-camera calibration method that will work in uncontrolled scenes. The long de…
SA-LIVO: Efficient LiDAR-Inertial-Visual Odometry with Subspace-Aware Degeneracy Handling
Key Laboratory of Space Active Opto-Electronics Technology, Shanghai Institute of Technical Physics, Chinese Academy of Sciences（中国科学院上海技术物理研究所空间主动光电技术重点实验室）；Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences（中国科学院大学杭州高等研究院）
FAST-LIVGO: A Degeneracy-Robust LiDAR-Inertial-Visual-GNSS Fusion Odometry
College of Mechatronics and Control Engineering, Shenzhen University（深圳大学机电与控制工程学院）；Department of Mechanical Engineering, The University of Hong Kong（香港大学机械工程系）；College of Automation, Harbin Engineering University（哈尔滨工程大学自动化学院）
To achieve accurate and robust pose estimation in Simultaneous Localization and Mapping (SLAM) task, multi-sensor fusion is proven to be an effective solution and thus provides great potential in robotic applications. This paper proposes FAST-LIVO, a fast LiDAR-Inertial-Visual Odometry system, which builds on two tightly-coupled and direct odometry subsystems: a VIO subsystem and a LIO subsystem. The LIO subsystem registers raw points (instead of feature points on e.g., edges or planes) of a new scan to an incrementally-built point cloud map. The map points are additionally attached with image…
FIRE-LIVWO: Robust LiDAR-Inertial-Visual-Wheel Odometry via Failure-Immune mmWave Radar Enhancement
School of Mechatronic Engineering, China University of Mining and Technology（中国矿业大学机电工程学院）；Jiangsu Collaborative Innovation Center of Intelligent Mining Equipment, China University of Mining and Technology（中国矿业大学江苏省智能采矿装备协同创新中心）；National Key Laboratory of Intelligent Mining Equipment Technology, China University of Mining and Technology（中国矿业大学智能采矿装备技术全国重点实验室）
LiDAR odometry and mapping (LOAM) has been playing an important role in autonomous vehicles, due to its ability to simultaneously localize the robot's pose and build high-precision, high-resolution maps of the surrounding environment. This enables autonomous navigation and safe path planning of autonomous vehicles. In this paper, we present a robust, real-time LOAM algorithm for LiDARs with small FoV and irregular samplings. By taking effort on both front-end and back-end, we address several fundamental challenges arising from such LiDARs, and achieve better performance in both precision and e…
A Numerically-Robust ROS 2 Port of iG-LIO: Diagnosing and Fixing Toolchain-Induced Failures in Incremental GICP LiDAR-Inertial Odometry
Robotics Institute (RI), Carnegie Mellon University (CMU)（卡内基梅隆大学机器人研究所）；Institute of Systems and Robotics (ISR), University of Coimbra (UC)（科英布拉大学系统与机器人研究所）
LiDAR-VGGT: Cross-Modal Coarse-to-Fine Fusion for Globally Consistent and Metric-Scale Dense Mapping
State Key Laboratory of Industrial Control Technology, Institute of Cyber-Systems and Control, Zhejiang University（工业控制技术国家重点实验室，系统与控制研究院，浙江大学）；Differential Robot Technology Co., Ltd.（差分机器人技术有限公司）；College of Intelligence Science and Technology, National University of Defense Technology（智能科学与技术学院，国防科技大学）
Benchmarking Visual Feature Representations for LiDAR-Inertial-Visual Odometry Under Challenging Conditions
Department of Covergence IT Engineering, Pohang University of Science and Technology(POSTECH)（convergence IT工程系，POSTECH大学）；School of Mechanical Engineering, Kunsan National University（机械工程学院，库山国立大学）；Department of Artificial Intelligence and Robotics, Sejong University（人工智能与机器人系，世宗大学）；Disaster Scientific Investigation Division Disaster Investigation Technology Team, National Disaster Management Research Institute, Ministry of the Interior and Safety（灾害科学调查 division 灾害调查技术团队，国家灾害管理研究所，内政与安全部）；Department of Electrical Engineering, POSTECH（电气工程系，POSTECH大学）
The Oxford Spires Dataset: Benchmarking Large-Scale LiDAR-Visual Localisation, Reconstruction and Radiance Field Methods
Oxford Robotics Inst., Dept. of Eng. Science, Univ. of Oxford, UK（牛津大学机器人研究所、工程科学系）；Group of Automation, Robotics and Computer Vision (AUROVA), University of Alicante, Spain（自动化、机器人与计算机视觉小组（AUROVA）、阿尔基兰特大学）
PINGS: Gaussian Splatting Meets Distance Fields within a Point-Based Implicit Neural Map
Center for Robotics, University of Bonn（博恩大学机器人中心）；MAVLab, TU Delft（代尔夫特理工大学MAVLab）；Lamarr Institute for Machine Learning and Artificial Intelligence（德国拉马尔人工智能与机器学习研究所）
Dahlem Center for Machine Learning and Robotics (DCMLR), Department of Mathematics and Computer Science, Freie Universität Berlin（达赫姆机器学习与机器人研究中心（DCMLR），数学与计算机科学系，柏林自由大学）
School of Geodesy and Geomatics, Wuhan University（武汉大学测绘学院）；INDRIVING Co.,Ltd（英驾科技股份有限公司）
With the ability of providing direct and accurate enough range measurements, light detection and ranging (LiDAR) is playing an essential role in localization and detection for autonomous vehicles. Since single LiDAR suffers from hardware failure and performance degradation intermittently, we present a multi-LiDAR integration scheme in this article. Our framework tightly couples multiple non-repetitive scanning LiDARs with inertial, encoder, and global navigation satellite system (GNSS) into pose estimation and simultaneous global map generation. Primarily, we formulate a precise synchronizatio…
EKF-Based Fusion of Wi-Fi/LiDAR/IMU for Indoor Localization and Navigation
School of Advanced Technology, Xi'an Jiaotong-Liverpool University（西安交通大学利物浦大学先进技术学院）；Department of Electrical Engineering and Electronics, University of Liverpool（利物浦大学电子工程系）；Zhejiang University of Technology（浙江工业大学）；School of Artificial Intelligence, Suzhou Vocational Institute of Industrial Technology（苏州工业职业技术学院人工智能学院）
LIT-GS: LiDAR-Inertial-Thermal Gaussian Splatting for Illumination-Robust Mapping
College of Mechatronics and Control Engineering, Shenzhen University（深圳大学机电与控制工程学院）；Department of Mechanical Engineering, The University of Hong Kong（香港大学机械工程系）
Institute for Aerospace Studies University of Toronto Toronto, Canada
In self-driving, standalone GPS is generally considered to have insufficient positioning accuracy to stay in lane. Instead, many turn to LIDAR localization, but this comes at the expense of building LIDAR maps that can be costly to maintain. Another possibility is to use semantic cues such as lane lines and traffic lights to achieve localization, but these are usually not continuously visible. This issue can be remedied by combining semantic cues with GPS to fill in the gaps. However, due to elapsed time between mapping and localization, the live GPS frame can be offset from the semantic map f…
DeepIPCv3: Event-Aware Multi-Modal Sensor Fusion for Sudden Pedestrian Crossing Avoidance
Department of Computer Science and Electronics, Universitas Gadjah Mada（计算机科学与电子系，加雅马达大学）；Department of Computer Science and Engineering, Toyohashi University of Technology（计算机科学与工程系，东福士大学）
A Vendor-Agnostic LiDAR Data Conversion System with Multi-Signal Detection and Multi-Format Output
BITS Pilani, Hyderabad Campus（比拉理工学院海德拉巴校区）；Department of Electrical and Electronics Engineering, BITS Pilani, Hyderabad Campus（比拉理工学院海德拉巴校区电气与电子工程系）；Department of Computer Science and Information Systems, BITS Pilani, Hyderabad Campus（比拉理工学院海德拉巴校区计算机科学与信息系统系）
Thin, reflective objects such as forks and whisks are common in our daily lives, but they are particularly challenging for robot perception because it is hard to reconstruct them using commodity RGB-D cameras or multi-view stereo techniques. While traditional pipelines struggle with objects like these, Neural Radiance Fields (NeRFs) have recently been shown to be remarkably effective for performing view synthesis on objects with thin structures or reflective materials. In this paper we explore the use of NeRF as a new source of supervision for robust robot vision systems. In particular, we dem…
Institute for Digital Technologies, Loughborough University（拉夫堡大学数字技术研究所）
Autonomous robots that assist humans in day to day living tasks are becoming increasingly popular. Autonomous mobile robots operate by sensing and perceiving their surrounding environment to make accurate driving decisions. A combination of several different sensors such as LiDAR, radar, ultrasound sensors and cameras are utilized to sense the surrounding environment of autonomous vehicles. These heterogeneous sensors simultaneously capture various physical attributes of the environment. Such multimodality and redundancy of sensing need to be positively utilized for reliable and consistent per…
