TCLC-GS：面向自动驾驶的紧耦合激光雷达-相机高斯泼溅
TCLC-GS: Tightly Coupled LiDAR-Camera Gaussian Splatting for Autonomous Driving
AI总结：
针对城市场景3D-GS方法未充分利用多传感器数据的问题，提出TCLC-GS紧耦合融合方法，通过混合显隐3D表示增强高斯属性，在两个自动驾驶数据集上实现SOTA性能与实时渲染。
摘要：
大多数针对城市场景的基于3D高斯泼溅（3D-GS）的方法直接使用3D激光雷达点初始化3D高斯，这不仅未充分利用激光雷达数据的能力，还忽略了融合激光雷达与相机数据的潜在优势。本文设计了一种新颖的紧耦合激光雷达-相机高斯泼溅方法（TCLC-GS），以充分利用激光雷达和相机两种传感器的综合优势，实现快速、高质量的3D重建以及新视角RGB/深度合成。TCLC-GS设计了一种由激光雷达-相机数据衍生的混合显式（彩色3D网格）与隐式（分层八叉树特征）3D表示，以丰富用于泼溅的3D高斯的属性。3D高斯的属性不仅与提供更完整3D形状和颜色信息的3D网格对齐初始化，还通过检索到的八叉树隐式特征被赋予更广泛的上下文信息。在高斯泼溅优化过程中，3D网格提供密集深度信息作为监督，通过学习鲁棒的几何结构来增强训练过程。在Waymo Open Dataset和nuScenes Dataset上开展的综合评估验证了我们方法的最优（SOTA）性能。使用单张NVIDIA RTX 3090 Ti，我们的方法展现出快速训练能力，在城市场景中，于1920x1280分辨率（Waymo）下实现90 FPS的实时RGB和深度渲染，于1600x900分辨率（nuScenes）下达到120 FPS。
英文摘要：
Most 3D Gaussian Splatting (3D-GS) based methods for urban scenes initialize 3D Gaussians directly with 3D LiDAR points, which not only underutilizes LiDAR data capabilities but also overlooks the potential advantages of fusing LiDAR with camera data. In this paper, we design a novel tightly coupled LiDAR-Camera Gaussian Splatting (TCLC-GS) to fully leverage the combined strengths of both LiDAR and camera sensors, enabling rapid, high-quality 3D reconstruction and novel view RGB/depth synthesis. TCLC-GS designs a hybrid explicit (colorized 3D mesh) and implicit (hierarchical octree feature) 3D representation derived from LiDAR-camera data, to enrich the properties of 3D Gaussians for splatting. 3D Gaussian's properties are not only initialized in alignment with the 3D mesh which provides more completed 3D shape and color information, but are also endowed with broader contextual information through retrieved octree implicit features. During the Gaussian Splatting optimization process, the 3D mesh offers dense depth information as supervision, which enhances the training process by learning of a robust geometry. Comprehensive evaluations conducted on the Waymo Open Dataset and nuScenes Dataset validate our method's state-of-the-art (SOTA) performance. Utilizing a single NVIDIA RTX 3090 Ti, our method demonstrates fast training and achieves real-time RGB and depth rendering at 90 FPS in resolution of 1920x1280 (Waymo), and 120 FPS in resolution of 1600x900 (nuScenes) in urban scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
