SeaSplat:利用三维高斯泼溅与物理基础图像形成模型表示水下场景
SeaSplat: Representing Underwater Scenes with 3D Gaussian Splatting and a Physically Grounded Image Formation Model
AI总结:
SeaSplat 将三维高斯泼溅与物理基础水下图像形成模型结合,实现水下场景实时渲染,在提升新视角渲染质量的同时恢复场景真实颜色并改善深度估计。
摘要:
我们提出 SeaSplat,一种利用三维辐射场最新进展实现水下场景实时渲染的方法。水下场景是极具挑战性的视觉环境,因为通过水这类介质进行渲染会在图像捕获过程中引入与距离和颜色相关的效应。我们将三维高斯泼溅(3DGS)——辐射场领域的一项最新进展,能够实现完整三维场景的快速训练和实时渲染——与一个物理基础的水下图像形成模型相结合。将 SeaSplat 应用于 SeaThru-NeRF 数据集中的真实世界场景(该场景由一台水下航行器在美属维尔京群岛采集)以及经仿真退化的真实世界场景,我们不仅在介质存在的情况下渲染场景新视角时获得了更高的定量性能,而且能够恢复场景底层真实颜色,并将渲染结果恢复为不存在中间介质的状态。我们表明,水下图像形成有助于学习场景结构,获得更好的深度图,同时证明我们的改进保持了利用三维高斯表示所带来的显著计算优势。
英文摘要:
We introduce SeaSplat, a method to enable real-time rendering of underwater scenes leveraging recent advances in 3D radiance fields. Underwater scenes are challenging visual environments, as rendering through a medium such as water introduces both range and color dependent effects on image capture. We constrain 3D Gaussian Splatting (3DGS), a recent advance in radiance fields enabling rapid training and real-time rendering of full 3D scenes, with a physically grounded underwater image formation model. Applying SeaSplat to the real-world scenes from SeaThru-NeRF dataset, a scene collected by an underwater vehicle in the US Virgin Islands, and simulation-degraded real-world scenes, not only do we see increased quantitative performance on rendering novel viewpoints from the scene with the medium present, but are also able to recover the underlying true color of the scene and restore renders to be without the presence of the intervening medium. We show that the underwater image formation helps learn scene structure, with better depth maps, as well as show that our improvements maintain the significant computational improvements afforded by leveraging a 3D Gaussian representation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。