GS-ROR$^2$: 双向引导的3DGS和SDF用于反射物体重光照与重建
GS-ROR$^2$: Bidirectional-guided 3DGS and SDF for Reflective Object Relighting and Reconstruction
AI总结：
提出一种双向引导框架，通过SDF辅助的高斯溅射优化重光照模型，并利用GS引导的SDF增强实现高质量几何重建，解决反射物体重光照与重建中的几何约束和细节捕捉问题。
摘要：
3D高斯溅射(3DGS)因其细致的表达能力和高效的渲染速度，在新视角合成方面展现出强大能力。然而，使用3DGS创建可重光照的3D资产并重建忠实几何仍然存在问题，特别是对于反射物体，其不连续表示给几何约束带来困难。体积符号距离场(SDF)方法提供了鲁棒的几何重建，但昂贵的射线步进阻碍了其实时应用并减慢了训练速度。此外，这些方法难以捕捉尖锐的几何细节。为此，我们提出以互补方式双向引导3DGS和SDF，包括SDF辅助的高斯溅射用于重光照模型的高效优化，以及GS引导的SDF增强用于高质量几何重建。SDF辅助高斯溅射的核心是混合高斯与SDF之间的深度和法线相互监督，避免了SDF昂贵的体积渲染。得益于这种相互监督，学习到的混合高斯以最小的时间成本得到良好约束。由于高斯以延迟着色模式渲染，alpha混合的高斯是平滑的，但单个高斯可能仍然是异常值，产生漂浮伪影。因此，我们引入SDF感知的剪枝策略，移除位于SDF定义表面远处的高斯异常值，避免漂浮问题。这样，我们的GS框架提供了合理的法线并实现了逼真的重光照，但来自深度的网格仍然存在问题。因此，我们设计了GS引导的SDF细化，利用来自高斯的混合法线微调SDF。通过这种增强，我们的方法可以以额外17%的训练时间为代价，为反射物体提供高质量的网格。
英文摘要：
3D Gaussian Splatting (3DGS) has shown a powerful capability for novel view synthesis due to its detailed expressive ability and highly efficient rendering speed. Unfortunately, creating relightable 3D assets and reconstructing faithful geometry with 3DGS is still problematic, particularly for reflective objects, as its discontinuous representation raises difficulties in constraining geometries. Volumetric signed distance field (SDF) methods provide robust geometry reconstruction, while the expensive ray marching hinders its real-time application and slows the training. Besides, these methods struggle to capture sharp geometric details. To this end, we propose to guide 3DGS and SDF bidirectionally in a complementary manner, including an SDF-aided Gaussian splatting for efficient optimization of the relighting model and a GS-guided SDF enhancement for high-quality geometry reconstruction. At the core of our SDF-aided Gaussian splatting is the mutual supervision of the depth and normal between blended Gaussians and SDF, which avoids the expensive volume rendering of SDF. Thanks to this mutual supervision, the learned blended Gaussians are well-constrained with a minimal time cost. As the Gaussians are rendered in a deferred shading mode, the alpha-blended Gaussians are smooth, while individual Gaussians may still be outliers, yielding floater artifacts. Therefore, we introduce an SDF-aware pruning strategy to remove Gaussian outliers located distant from the surface defined by SDF, avoiding floater issue. This way, our GS framework provides reasonable normal and achieves realistic relighting, while the mesh from depth is still problematic. Therefore, we design a GS-guided SDF refinement, which utilizes the blended normal from Gaussians to finetune SDF. With this enhancement, our method can further provide high-quality meshes for reflective objects at the cost of 17% extra training time.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
