VIRGi: 基于3D高斯溅射的视图依赖性即时重绘
VIRGi: View-dependent Instant Recoloring of 3D Gaussians Splats
AI总结：
VIRGi通过分离颜色为漫反射和视图依赖成分，实现对3DGS场景的快速重绘，提升实时交互与视图依赖效果控制。
摘要：
3D高斯溅射（3DGS）最近通过其能够准确建模复杂3D场景的能力以及前所未有的渲染性能，转变了新型视图合成和3D重建领域。然而，仍然存在重大挑战：缺乏一种高效且逼真的方法来编辑场景内容的外观。在本文中，我们介绍了VIRGi，一种新的方法，用于快速编辑由3DGS建模的场景的颜色，同时保留视图依赖效果，如镜面高光。我们方法的关键在于一种新的架构，将颜色分为漫反射和视图依赖成分，并且一种多视图训练策略，整合了多个视角的图像块。相比传统的单视角批量训练，我们的3DGS表示提供了更精确的重建，并为重绘任务提供了坚实的基础。对于3DGS重绘，我们引入了一种快速方案，只需一个手动编辑的场景图像。通过微调单个MLP的权重，以及一个用于单次分割可编辑区域的模块，颜色编辑可以无缝传播到整个场景，仅需两秒钟，从而实现实时交互，并提供对视图依赖效果强度的控制。在各种数据集上的全面验证显示，基于神经辐射场表示的竞争对手在定量和定性方面均取得了显著的进步。
英文摘要：
3D Gaussian Splatting (3DGS) has recently transformed the fields of novel view synthesis and 3D reconstruction due to its ability to accurately model complex 3D scenes and its unprecedented rendering performance. However, a significant challenge persists: the absence of an efficient and photorealistic method for editing the appearance of the scene's content. In this paper we introduce VIRGi, a novel approach for rapidly editing the color of scenes modeled by 3DGS while preserving view-dependent effects such as specular highlights. Key to our method are a novel architecture that separates color into diffuse and view-dependent components, and a multi-view training strategy that integrates image patches from multiple viewpoints. Improving over the conventional single-view batch training, our 3DGS representation provides more accurate reconstruction and serves as a solid representation for the recoloring task. For 3DGS recoloring, we then introduce a rapid scheme requiring only one manually edited image of the scene from the end-user. By fine-tuning the weights of a single MLP, alongside a module for single-shot segmentation of the editable area, the color edits are seamlessly propagated to the entire scene in just two seconds, facilitating real-time interaction and providing control over the strength of the view-dependent effects. An exhaustive validation on diverse datasets demonstrates significant quantitative and qualitative advancements over competitors based on Neural Radiance Fields representations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
