arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.19233cs.GR
收藏

UniMGS: 统一网格和3D高斯点散布的单次扫描转换与基于代理的变形

UniMGS: Unifying Mesh and 3D Gaussian Splatting with Single-Pass Rasterization and Proxy-Based Deformation

College of Intelligent Robotics and Advanced Manufacturing, Fudan University(复旦大学智能机器人与先进制造学院);Fysics Intelligence Technologies Co., Ltd.(菲视智能科技有限公司);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学)

Zeyu Xiao, Mingyang Sun, Yimin Cong, Lintao Wang, Dongliang Kou, Zhenyi Wu, Dingkang Yang, Peng Zhai, Zeyu Wang, Lihua Zhang

AI总结:

UniMGS通过统一框架实现网格与3D高斯点散布的单次反走样化渲染,解决遮挡和透明度问题,提升视觉效果。

摘要:

网格和3D高斯点散布(3DGS)的联合渲染和变形在图形应用中具有重要价值,因为这两种表示方法各自提供了互补的优势。然而,由于表示和渲染管线的不同,现有研究分别渲染网格和3DGS,使得难以准确处理遮挡和透明度问题。此外,变形后的3DGS仍然因对代理网格拓扑质量的敏感性而产生视觉伪影。这些问题严重阻碍了3DGS和网格的联合使用,使得将3DGS适应到传统的网格导向图形管线变得困难。我们提出了UniMGS,这是首个能够单次反走样化地对网格和3DGS进行扫描转换的统一框架,其采用基于代理网格的新型绑定策略用于3DGS的变形。我们的核心见解是通过反走样化的α混合在单次扫描中混合三角形和高斯碎片的颜色,从而实现视觉上一致的结果,并精确处理遮挡和透明度问题。为了提高变形后3DGS的视觉效果,我们的以高斯为中心的绑定策略采用代理网格并空间关联高斯分布与网格面,显著减少了渲染伪影。通过这两个组件,UniMGS使在统一框架内可视化和操作由网格或3DGS表示的3D对象成为可能,为具身人工智能、虚拟现实和游戏等领域开辟了新的可能性。我们将发布源代码以促进未来研究。

英文摘要:

Joint rendering and deformation of mesh and 3D Gaussian Splatting (3DGS) have significant value as both representa tions offer complementary advantages for graphics applica tions. However, due to differences in representation and ren dering pipelines, existing studies render meshes and 3DGS separately, making it difficult to accurately handle occlusions and transparency. Moreover, the deformed 3DGS still suffers from visual artifacts due to the sensitivity to the topology quality of the proxy mesh. These issues pose serious obsta cles to the joint use of 3DGS and meshes, making it diffi cult to adapt 3DGS to conventional mesh-oriented graphics pipelines. We propose UniMGS, the first unified framework for rasterizing mesh and 3DGS in a single-pass anti-aliased manner, with a novel binding strategy for 3DGS deformation based on proxy mesh. Our key insight is to blend the col ors of both triangle and Gaussian fragments by anti-aliased α-blending in a single pass, achieving visually coherent re sults with precise handling of occlusion and transparency. To improve the visual appearance of the deformed 3DGS, our Gaussian-centric binding strategy employs a proxy mesh and spatially associates Gaussians with the mesh faces, signifi cantly reducing rendering artifacts. With these two compo nents, UniMGS enables the visualization and manipulation of 3D objects represented by mesh or 3DGS within a unified framework, opening up new possibilities in embodied AI, vir tual reality, and gaming. We will release our source code to facilitate future research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑