arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.16503cs.CVeess.IV

HDRSplat:基于原始图像的用于高动态范围3D场景重建的高斯泼溅

HDRSplat: Gaussian Splatting for High Dynamic Range 3D Scene Reconstruction from Raw Images

Indian Institute of Technology, Madras(马德拉斯印度理工学院)

Shreyas Singh, Aryan Garg, Kaushik Mitra

AI总结:

HDRSplat提出直接基于14-bit线性raw图像训练3DGS,通过线性HDR损失函数和光栅化调优克服点云初始化敏感性,实现快速且高保真的高动态范围3D场景重建。

摘要:

3D Gaussian Splatting (3DGS) 的近期出现彻底改变了3D场景重建领域,实现了高保真的实时新视角合成。然而,除了 RawNeRF 之外,所有先前的 3DGS 和基于 NeRF 的方法都依赖于 8-bit 色调映射的低动态范围 (LDR) 图像进行场景重建。此类方法在需要更高动态范围的场景中难以实现准确重建。例如在夜间或光线不足的室内空间(具有低信噪比)拍摄的场景,以及具有极端对比度阴影区域的日光场景。我们提出的方法 HDRSplat 对 3DGS 进行了定制,使其能够直接在近黑暗环境下的 14-bit 线性 raw 图像上进行训练,从而保留了场景的完整动态范围和内容。我们的核心贡献有两方面:首先,我们提出了一种适用于线性 HDR 空间的损失函数,能够同时从嘈杂的暗区和近乎饱和的亮区有效提取场景信息,同时在不增加球谐函数阶数的情况下处理视角相关颜色。其次,通过精细的光栅化调优,我们隐式地克服了 3DGS 对点云初始化的严重依赖和敏感性。这对于低纹理、高景深和低照度区域的准确重建至关重要。HDRSplat 是迄今为止最快的 14-bit (HDR) 3D 场景重建方法,每场景耗时 $\le$15 分钟(比先前最先进的 RawNeRF 快约 30 倍)。它还拥有最快的推理速度,达到 $\ge$120fps。我们进一步展示了 HDR 场景重建的适用性,展示了合成散焦、稠密深度图提取,以及曝光、色调映射和视角的拍摄后控制等各种应用。

英文摘要:

The recent advent of 3D Gaussian Splatting (3DGS) has revolutionized the 3D scene reconstruction space enabling high-fidelity novel view synthesis in real-time. However, with the exception of RawNeRF, all prior 3DGS and NeRF-based methods rely on 8-bit tone-mapped Low Dynamic Range (LDR) images for scene reconstruction. Such methods struggle to achieve accurate reconstructions in scenes that require a higher dynamic range. Examples include scenes captured in nighttime or poorly lit indoor spaces having a low signal-to-noise ratio, as well as daylight scenes with shadow regions exhibiting extreme contrast. Our proposed method HDRSplat tailors 3DGS to train directly on 14-bit linear raw images in near darkness which preserves the scenes' full dynamic range and content. Our key contributions are two-fold: Firstly, we propose a linear HDR space-suited loss that effectively extracts scene information from noisy dark regions and nearly saturated bright regions simultaneously, while also handling view-dependent colors without increasing the degree of spherical harmonics. Secondly, through careful rasterization tuning, we implicitly overcome the heavy reliance and sensitivity of 3DGS on point cloud initialization. This is critical for accurate reconstruction in regions of low texture, high depth of field, and low illumination. HDRSplat is the fastest method to date that does 14-bit (HDR) 3D scene reconstruction in $\le$15 minutes/scene ($\sim$30x faster than prior state-of-the-art RawNeRF). It also boasts the fastest inference speed at $\ge$120fps. We further demonstrate the applicability of our HDR scene reconstruction by showcasing various applications like synthetic defocus, dense depth map extraction, and post-capture control of exposure, tone-mapping and view-point.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑