基于网格的高斯溅射用于实时大规模形变
Mesh-based Gaussian Splatting for Real-time Large-scale Deformation
AI总结：
提出基于网格的高斯溅射表示，实现实时大规模形变，通过网格与高斯绑定及形变技术，实现高质量重建与有效形变，平均帧率65 FPS。
摘要：
神经隐式表示，包括神经距离场和神经辐射场，已展现出重建具有复杂几何和拓扑结构的表面以及生成场景新视角的显著能力。然而，用户直接以实时方式对这些隐式表示进行大规模形变或操作仍具挑战性。高斯溅射（GS）近来成为一种具有显式几何的、用于表示静态场景并促进高质量实时新视角合成的有前景方法。然而，由于使用离散高斯且缺乏显式拓扑，它难以被形变。为解决此问题，我们开发了一种新颖的基于GS的方法，支持交互式形变。我们的关键思想是设计一种创新的基于网格的GS表示，并将其集成到高斯学习和操作中。3D高斯定义在显式网格上，并相互绑定：3D高斯的渲染引导网格面分裂以实现自适应细化，而网格面分裂则引导3D高斯的分裂。此外，显式网格约束有助于正则化高斯分布，抑制质量差的高斯（例如错位的高斯、长窄形高斯），从而在形变过程中增强视觉质量并避免伪影。基于此表示，我们进一步引入大规模高斯形变技术以实现可形变GS，该技术根据关联网格的操作改变3D高斯的参数。我们的方法受益于现有网格形变数据集，以实现更真实的数据驱动高斯形变。大量实验表明，我们的方法实现了高质量重建和有效形变，同时以高帧率（平均65 FPS）保持有前景的渲染结果。
英文摘要：
Neural implicit representations, including Neural Distance Fields and Neural Radiance Fields, have demonstrated significant capabilities for reconstructing surfaces with complicated geometry and topology, and generating novel views of a scene. Nevertheless, it is challenging for users to directly deform or manipulate these implicit representations with large deformations in the real-time fashion. Gaussian Splatting(GS) has recently become a promising method with explicit geometry for representing static scenes and facilitating high-quality and real-time synthesis of novel views. However,it cannot be easily deformed due to the use of discrete Gaussians and lack of explicit topology. To address this, we develop a novel GS-based method that enables interactive deformation. Our key idea is to design an innovative mesh-based GS representation, which is integrated into Gaussian learning and manipulation. 3D Gaussians are defined over an explicit mesh, and they are bound with each other: the rendering of 3D Gaussians guides the mesh face split for adaptive refinement, and the mesh face split directs the splitting of 3D Gaussians. Moreover, the explicit mesh constraints help regularize the Gaussian distribution, suppressing poor-quality Gaussians(e.g. misaligned Gaussians,long-narrow shaped Gaussians), thus enhancing visual quality and avoiding artifacts during deformation. Based on this representation, we further introduce a large-scale Gaussian deformation technique to enable deformable GS, which alters the parameters of 3D Gaussians according to the manipulation of the associated mesh. Our method benefits from existing mesh deformation datasets for more realistic data-driven Gaussian deformation. Extensive experiments show that our approach achieves high-quality reconstruction and effective deformation, while maintaining the promising rendering results at a high frame rate(65 FPS on average).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
