基于参考的高斯溅射可控场景风格化
Reference-based Controllable Scene Stylization with Gaussian Splatting
AI总结：
针对现有参考式场景风格化方法依赖NeRF、体渲染速度慢的问题，提出ReGS方法，通过纹理引导的高斯几何调整机制实现3DGS的实时可控风格化，效果达SOTA且支持自由视角实时渲染。
摘要：
基于内容对齐参考图像编辑外观的参考式场景风格化是一个新兴研究领域。现有方法通常以预训练的神经辐射场（NeRF）为基础，学习与给定风格匹配的全新外观。这类方法虽有效果，但本质上受困于耗时的体渲染，因此在许多实时应用中并不实用。在本研究中，我们提出ReGS方法，将3D高斯溅射（3D Gaussian Splatting, 3DGS）适配于参考式风格化，以实现实时风格化视图合成。编辑预训练3DGS的外观颇具挑战性，因为它采用离散高斯作为3D表示，将外观与几何结构紧密绑定。若像现有方法那样仅优化外观，通常不足以建模给定参考图像中的连续纹理。为应对这一挑战，我们提出一种新颖的纹理引导控制机制，可自适应调整局部负责的高斯体至新的几何排布，以匹配所需的纹理细节。该过程由纹理线索引导以实现高效外观编辑，并通过场景深度进行正则化以保留原始几何结构。借助这些新颖设计，我们证明ReGS能够生成达到当前最优水平的风格化结果，既贴合参考纹理，又能在自由视角导航中实现实时渲染速度。
英文摘要：
Referenced-based scene stylization that edits the appearance based on a content-aligned reference image is an emerging research area. Starting with a pretrained neural radiance field (NeRF), existing methods typically learn a novel appearance that matches the given style. Despite their effectiveness, they inherently suffer from time-consuming volume rendering, and thus are impractical for many real-time applications. In this work, we propose ReGS, which adapts 3D Gaussian Splatting (3DGS) for reference-based stylization to enable real-time stylized view synthesis. Editing the appearance of a pretrained 3DGS is challenging as it uses discrete Gaussians as 3D representation, which tightly bind appearance with geometry. Simply optimizing the appearance as prior methods do is often insufficient for modeling continuous textures in the given reference image. To address this challenge, we propose a novel texture-guided control mechanism that adaptively adjusts local responsible Gaussians to a new geometric arrangement, serving for desired texture details. The proposed process is guided by texture clues for effective appearance editing, and regularized by scene depth for preserving original geometric structure. With these novel designs, we show ReGs can produce state-of-the-art stylization results that respect the reference texture while embracing real-time rendering speed for free-view navigation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
