arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.10050cs.CV

Texture-GS:解耦三维高斯溅射的几何与纹理以支持编辑

Texture-GS: Disentangling the Geometry and Texture for 3D Gaussian Splatting Editing

Tsinghua University(清华大学);Tencent AI Lab(腾讯AI实验室);Cardiff University(卡迪夫大学)

Tian-Xing Xu, Wenbo Hu, Yu-Kun Lai, Ying Shan, Song-Hai Zhang

AI总结:

针对三维高斯溅射中外观与几何耦合导致编辑困难的问题,提出Texture-GS,通过纹理映射模块将外观解耦为二维纹理,实现高保真外观编辑并支持消费级设备实时渲染。

摘要:

三维高斯溅射作为一种开创性的方法,因其高保真重建和实时渲染能力而日益受到关注。然而,它将场景的外观与几何耦合在高斯属性中,这阻碍了编辑操作(如纹理交换)的灵活性。为解决这一问题,我们提出了一种新方法,即Texture-GS,通过将外观表示为映射到三维表面上的二维纹理,从而将外观与几何解耦,进而促进外观编辑。技术上,这种解耦是通过我们提出的纹理映射模块实现的,该模块包含一个用于学习三维高斯中心UV坐标的UV映射MLP、一个该MLP的局部泰勒展开以高效近似射线-高斯交点处的UV坐标,以及一个可学习的纹理以捕捉细粒度的外观。在DTU数据集上的大量实验表明,我们的方法不仅促进了高保真外观编辑,还在消费级设备(如单个RTX 2080 Ti GPU)上实现了实时渲染。

英文摘要:

3D Gaussian splatting, emerging as a groundbreaking approach, has drawn increasing attention for its capabilities of high-fidelity reconstruction and real-time rendering. However, it couples the appearance and geometry of the scene within the Gaussian attributes, which hinders the flexibility of editing operations, such as texture swapping. To address this issue, we propose a novel approach, namely Texture-GS, to disentangle the appearance from the geometry by representing it as a 2D texture mapped onto the 3D surface, thereby facilitating appearance editing. Technically, the disentanglement is achieved by our proposed texture mapping module, which consists of a UV mapping MLP to learn the UV coordinates for the 3D Gaussian centers, a local Taylor expansion of the MLP to efficiently approximate the UV coordinates for the ray-Gaussian intersections, and a learnable texture to capture the fine-grained appearance. Extensive experiments on the DTU dataset demonstrate that our method not only facilitates high-fidelity appearance editing but also achieves real-time rendering on consumer-level devices, e.g. a single RTX 2080 Ti GPU.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑