用于3D生成的四面体泼溅
Tetrahedron Splatting for 3D Generation
AI总结：
提出Tetrahedron Splatting (TeT-Splatting)这一新型3D表示，通过在结构化四面体网格中集成基于表面的体积渲染和可微光栅化器，结合正则化项，在无需网格提取即可优化收敛的同时，实现了精确网格提取与实时渲染，在3D生成中取得更优权衡。
摘要：
3D表示对于借助2D扩散先验推动3D生成的显著进步至关重要。作为一种灵活的表示方法，NeRF首先被采用于3D表示。然而，基于密度的体积渲染使其同时面临密集的计算开销和不准确的网格提取问题。使用有符号距离场和Marching Tetrahedra，DMTet能够实现精确的网格提取和实时渲染，但在处理网格的大拓扑变化时存在局限，导致优化挑战。或者，3D Gaussian Splatting (3DGS) 在训练和渲染效率上均受青睐，但在网格提取方面表现不足。在本工作中，我们引入了一种新颖的3D表示，即Tetrahedron Splatting (TeT-Splatting)，它同时支持优化过程中的易收敛性、精确的网格提取和实时渲染。这是通过在结构化四面体网格内集成基于表面的体积渲染，同时保留精确网格提取的期望能力，以及一个基于瓦片的可微四面体光栅化器来实现的。此外，我们为有符号距离场引入了Eikonal和法线一致性正则化项，以提高生成质量和稳定性。关键的是，我们的表示可以在不提取网格的情况下进行训练，使优化过程更容易收敛。我们的TeT-Splatting可以轻松集成到现有的3D生成流程中，并配合多边形网格进行纹理优化。大量实验表明，与以往的方法相比，在不同的3D生成设置下，我们的TeT-Splatting在收敛速度、渲染效率和网格质量之间取得了更优的权衡。
英文摘要：
3D representation is essential to the significant advance of 3D generation with 2D diffusion priors. As a flexible representation, NeRF has been first adopted for 3D representation. With density-based volumetric rendering, it however suffers both intensive computational overhead and inaccurate mesh extraction. Using a signed distance field and Marching Tetrahedra, DMTet allows for precise mesh extraction and real-time rendering but is limited in handling large topological changes in meshes, leading to optimization challenges. Alternatively, 3D Gaussian Splatting (3DGS) is favored in both training and rendering efficiency while falling short in mesh extraction. In this work, we introduce a novel 3D representation, Tetrahedron Splatting (TeT-Splatting), that supports easy convergence during optimization, precise mesh extraction, and real-time rendering simultaneously. This is achieved by integrating surface-based volumetric rendering within a structured tetrahedral grid while preserving the desired ability of precise mesh extraction, and a tile-based differentiable tetrahedron rasterizer. Furthermore, we incorporate eikonal and normal consistency regularization terms for the signed distance field to improve generation quality and stability. Critically, our representation can be trained without mesh extraction, making the optimization process easier to converge. Our TeT-Splatting can be readily integrated in existing 3D generation pipelines, along with polygonal mesh for texture optimization. Extensive experiments show that our TeT-Splatting strikes a superior tradeoff among convergence speed, render efficiency, and mesh quality as compared to previous alternatives under varying 3D generation settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
