HoloGS:基于 Microsoft HoloLens 2 的即时深度三维高斯泼溅
HoloGS: Instant Depth-based 3D Gaussian Splatting with Microsoft HoloLens 2
AI总结:
本文提出 HoloGS 工作流,利用 Microsoft HoloLens 2 即时获取图像、相机位姿和深度点云,绕过运动恢复结构预处理,直接实现三维高斯泼溅,并在室外雕像和室内植物场景中验证了渲染质量与几何精度。
摘要:
在摄影测量、计算机视觉和计算机图形学领域,神经三维场景重建任务推动了多种技术的探索。其中,三维高斯泼溅(3D Gaussian Splatting)因其使用三维高斯显式表示场景而脱颖而出,使其在三维点云提取和表面重建等任务中具有吸引力。受其潜力的激励,我们针对三维场景重建领域展开研究,旨在利用 Microsoft HoloLens 2 的能力实现即时三维高斯泼溅。我们提出了 HoloGS,一种利用 HoloLens 传感器数据的新颖工作流程,通过即时获取所需输入数据(即图像、相机位姿和来自深度感知的点云),绕过了运动恢复结构(Structure from Motion)等预处理步骤。我们提供了全面的研究,包括训练过程和渲染质量(通过峰值信噪比评估),以及由高斯中心稠密化得到的点云的几何三维精度(通过 Chamfer Distance 测量)。我们在两个自采集场景上评估了我们的方法:一个文化遗产雕像的室外场景和一个精细结构植物的室内场景。结果表明,HoloGS 所使用的 HoloLens 数据,包括 RGB 图像、对应的相机位姿以及用于初始化高斯的基于深度感知的点云,适合作为三维高斯泼溅的输入。
英文摘要:
In the fields of photogrammetry, computer vision and computer graphics, the task of neural 3D scene reconstruction has led to the exploration of various techniques. Among these, 3D Gaussian Splatting stands out for its explicit representation of scenes using 3D Gaussians, making it appealing for tasks like 3D point cloud extraction and surface reconstruction. Motivated by its potential, we address the domain of 3D scene reconstruction, aiming to leverage the capabilities of the Microsoft HoloLens 2 for instant 3D Gaussian Splatting. We present HoloGS, a novel workflow utilizing HoloLens sensor data, which bypasses the need for pre-processing steps like Structure from Motion by instantly accessing the required input data i.e. the images, camera poses and the point cloud from depth sensing. We provide comprehensive investigations, including the training process and the rendering quality, assessed through the Peak Signal-to-Noise Ratio, and the geometric 3D accuracy of the densified point cloud from Gaussian centers, measured by Chamfer Distance. We evaluate our approach on two self-captured scenes: An outdoor scene of a cultural heritage statue and an indoor scene of a fine-structured plant. Our results show that the HoloLens data, including RGB images, corresponding camera poses, and depth sensing based point clouds to initialize the Gaussians, are suitable as input for 3D Gaussian Splatting.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。