Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materials
TU Munich（慕尼黑工业大学）；Meta
Yawar Siddiqui, Tom Monnier, Filippos Kokkinos, Mahendra Kariya, Yanir Kleiman, Emilien Garreau, Oran Gafni, Natalia Neverova, Andrea Vedaldi, Roman Shapovalov, David Novotny
本文提出 AssetGen 文本到 3D 生成框架，通过分解多视图外观、PBR 材质重建、SDF 形状监督及 UV 纹理细化，显著提升几何与材质质量。
摘要：
我们提出 Meta 3D AssetGen（AssetGen），这是文本到 3D 生成领域的一项重要进展，可生成忠实、高质量的网格，并具备纹理和材质控制能力。与将阴影烘焙到 3D 对象外观中的工作相比，AssetGen 输出基于物理的渲染（PBR）材质，支持逼真的重新照明。AssetGen 首先生成对象的若干视图，并将着色外观通道与反照率外观通道分解，随后在 3D 中重建颜色、金属度和粗糙度，同时使用延迟着色损失实现高效监督。它还使用符号距离函数更可靠地表示 3D 形状，并引入相应损失进行直接形状监督。该方法通过融合内核实现，以获得较高的内存效率。在网格提取之后，一个在 UV 空间中运行的纹理细化 transformer 显著提升了清晰度和细节。在少视图重建方面，AssetGen 的 Chamfer Distance 相比最佳同期工作改善 17%，LPIPS 改善 40%；在与速度相当的最佳行业竞争者（包括支持 PBR 的方法）相比时，其获得 72% 的人类偏好。包含生成资产的项目页面位于 https://assetgen.github.io。
英文摘要：
We present Meta 3D AssetGen (AssetGen), a significant advancement in text-to-3D generation which produces faithful, high-quality meshes with texture and material control. Compared to works that bake shading in the 3D object's appearance, AssetGen outputs physically-based rendering (PBR) materials, supporting realistic relighting. AssetGen generates first several views of the object with factored shaded and albedo appearance channels, and then reconstructs colours, metalness and roughness in 3D, using a deferred shading loss for efficient supervision. It also uses a sign-distance function to represent 3D shape more reliably and introduces a corresponding loss for direct shape supervision. This is implemented using fused kernels for high memory efficiency. After mesh extraction, a texture refinement transformer operating in UV space significantly improves sharpness and details. AssetGen achieves 17% improvement in Chamfer Distance and 40% in LPIPS over the best concurrent work for few-view reconstruction, and a human preference of 72% over the best industry competitors of comparable speed, including those that support PBR. Project page with generated assets: https://assetgen.github.io
TexGaussian: Generating High-quality PBR Material via Octree-based 3D Gaussian Splatting
Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机研究所）；Baidu VIS（百度VIS）；Institute of Medical Technology, Peking University（北京大学医学技术研究所）
We introduce Meta 3D Gen (3DGen), a new state-of-the-art, fast pipeline for text-to-3D asset generation. 3DGen offers 3D asset creation with high prompt fidelity and high-quality 3D shapes and textures in under a minute. It supports physically-based rendering (PBR), necessary for 3D asset relighting in real-world applications. Additionally, 3DGen supports generative retexturing of previously generated (or artist-created) 3D shapes using additional textual inputs provided by the user. 3DGen integrates key technical components, Meta 3D AssetGen and Meta 3D TextureGen, that we developed for text-…
Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Rendering
University of Tübingen（蒂宾根大学）；Tübingen AI Center（蒂宾根人工智能中心）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；POSTECH（浦项科技大学）；Institute for Convergence Research and Education in Advanced Technology, Yonsei University（延世大学尖端技术融合研究与教育学院）
Zhejiang University（浙江大学）；Shanghai Innovation Institute（上海创新研究院）；Westlake University（西湖大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；Fudan University（复旦大学）
MaterialMVP: Illumination-Invariant Material Generation via Multi-view PBR Diffusion
Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；Tencent Hunyuan（腾讯文元）；The Hong Kong University of Science and Technology（香港科技大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））
PacTure: Efficient PBR Texture Generation on Packed Views with Visual Autoregressive Models
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（信息处理国家重点实验室，计算机学院，北京大学）；National Engineering Research Center of Visual Technology, School of Computer Science, Peking University（视觉技术国家工程研究中心，计算机学院，北京大学）；The Hong Kong University of Science and Technology（香港科技大学）；Light Illusions；PKU-AI 2 Robotics Joint Lab of Embodied AI（北京大学人工智能2机器人联合实验室）
DreamLifting: A Plug-in Module Lifting MV Diffusion Models for 3D Asset Generation
PCA Lab, VCIP, College of Computer Science, Nankai University（南开大学计算机学院VCIP实验室PCA Lab）；School of Intelligence Science and Technology, Nanjing University（南京大学智能科学与技术学院）；Horizon Robotics（地平线机器人）；D-Robotics
GeoSplatting: Towards Geometry Guided Gaussian Splatting for Physically-based Inverse Rendering
School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）；School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Wangxuan Institute of Computer Technology, Peking University（北京大学王轩计算机技术研究所）；State Key Laboratory of Multimedia Information Processing, Peking University（北京大学多媒体信息处理国家重点实验室）；State Key Laboratory of General Artificial Intelligence, Peking University（北京大学通用人工智能国家重点实验室）
