TaoTex：提升原生3D材质生成的纹理细节保真度
TaoTex: Boosting Texture Detail Fidelity for Native 3D Material Generation
- Alibaba Group（阿里巴巴集团）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
TaoTex是一种基于扩散的原生3D材质生成模型，通过数据构建智能体、多级特征融合模块、潜在到像素损失过渡及可学习视角嵌入，显著提升了单视角和多视角下的纹理细节重建保真度。
AI中文摘要：
近期的3D生成模型能够生成精确的几何形状，但在重建精细纹理方面仍存在困难。我们提出了一种基于扩散的原生3D材质生成模型TaoTex，通过定制策略和改进，忠实地恢复复杂纹理。首先，我们开发了一个数据构建智能体来创建高频纹理的3D资产，以弥合公开数据集中的数据缺口。使用这些数据进行训练显著增强了TaoTex恢复诸如文字和图案等具有挑战性细节的能力。其次，我们设计了一个多级特征融合（MLFF）模块，自适应地整合条件输入的局部和全局特征，为扩散模型提供更完整的纹理线索，从而增强重建保真度。为了减轻VAE重建误差，我们采用了从潜在空间到像素空间的损失过渡，进一步改善了像素级细节和生成质量。最后，我们通过引入可学习的视角嵌入，将TaoTex扩展到多视角输入，实现了跨视角准确且一致的材质重建。大量实验表明，我们的方法在单视角和多视角设置下，在保留纹理细节方面显著优于现有方法。
英文摘要：
Recent 3D generation models can produce accurate geometries while still struggling to reconstruct detailed textures. We propose a diffusion-based native 3D material generation model TaoTex, which faithfully recovers intricate textures through tailored strategies and improvements. First, we develop a data construction agent to create high-frequency textured 3D assets to bridge the data gap in public datasets. Training with these data significantly enhances the ability of TaoTex to recover challenging details such as text and patterns. Second, we design a multi-level feature fusion (MLFF) module to adaptively integrate local and global features of the conditional input, providing more complete texture cues for the diffusion model and thereby enhancing reconstruction fidelity. To alleviate VAE reconstruction errors, we adopt a latent-to-pixel space loss transition, further improving the pixel-level details and generation quality. Finally, we scale TaoTex to multi-view inputs by incorporating learnable viewpoint embeddings, achieving accurate and consistent material reconstruction across views. Extensive experiments demonstrate that our method significantly outperforms existing approaches in preserving texture details in both single- and multi-view settings.
