MeshCraft：探索基于流的DiT实现高效可控的网格生成
MeshCraft: Exploring Efficient and Controllable Mesh Generation with Flow-based DiTs
AI总结：
MeshCraft提出基于流的扩散Transformer与VAE结合，实现高效可控的三维网格生成，显著提升速度并支持预定义面数。
摘要：
在三维内容创作领域，通过人工智能模型实现最优网格拓扑结构一直是三维艺术家长期追求的目标。以往的方法，如MeshGPT，已探索了通过网格自回归技术生成可直接使用的三维对象。尽管这些方法产生了视觉上令人印象深刻的结果，但它们依赖于自回归过程中的逐标记预测，导致了一些显著的限制，包括极慢的生成速度和不可控的网格面数。在本文中，我们介绍了MeshCraft，一个用于高效且可控网格生成的新颖框架，它利用连续空间扩散来生成离散的三角面。具体来说，MeshCraft由两个核心组件组成：1）一个基于Transformer的变分自编码器（VAE），将原始网格编码为连续的面级标记，并将其解码回原始网格；2）一个以面数为条件的基于流的扩散Transformer，能够生成具有预定义面数的高质量三维网格。通过利用扩散模型同时生成整个网格拓扑，MeshCraft相比自回归方法实现了显著更快速度的高保真网格生成。具体而言，MeshCraft仅需3.2秒即可生成一个800面的网格（比现有基线快35倍）。大量实验表明，MeshCraft在ShapeNet数据集上的定性和定量评估中均优于最先进的技术，并在Objaverse数据集上展现出优越的性能。此外，它与现有的条件引导策略无缝集成，展示了其减轻艺术家在网格创建中耗时的手动工作的潜力。
英文摘要：
In the domain of 3D content creation, achieving optimal mesh topology through AI models has long been a pursuit for 3D artists. Previous methods, such as MeshGPT, have explored the generation of ready-to-use 3D objects via mesh auto-regressive techniques. While these methods produce visually impressive results, their reliance on token-by-token predictions in the auto-regressive process leads to several significant limitations. These include extremely slow generation speeds and an uncontrollable number of mesh faces. In this paper, we introduce MeshCraft, a novel framework for efficient and controllable mesh generation, which leverages continuous spatial diffusion to generate discrete triangle faces. Specifically, MeshCraft consists of two core components: 1) a transformer-based VAE that encodes raw meshes into continuous face-level tokens and decodes them back to the original meshes, and 2) a flow-based diffusion transformer conditioned on the number of faces, enabling the generation of high-quality 3D meshes with a predefined number of faces. By utilizing the diffusion model for the simultaneous generation of the entire mesh topology, MeshCraft achieves high-fidelity mesh generation at significantly faster speeds compared to auto-regressive methods. Specifically, MeshCraft can generate an 800-face mesh in just 3.2 seconds (35$\times$ faster than existing baselines). Extensive experiments demonstrate that MeshCraft outperforms state-of-the-art techniques in both qualitative and quantitative evaluations on ShapeNet dataset and demonstrates superior performance on Objaverse dataset. Moreover, it integrates seamlessly with existing conditional guidance strategies, showcasing its potential to relieve artists from the time-consuming manual work involved in mesh creation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
