arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.12928cs.CV
收藏

DreamCraft3D++:基于多平面重建模型的高效分层3D生成

DreamCraft3D++: Efficient Hierarchical 3D Generation with Multi-Plane Reconstruction Model

Tsinghua University(清华大学);VAST;Zhejiang University(浙江大学)

Jingxiang Sun, Cheng Peng, Ruizhi Shao, Yuan-Chen Guo, Xiaochen Zhao, Yangguang Li, Yanpei Cao, Bo Zhang, Yebin Liu

AI总结:

DreamCraft3D++通过引入前馈式多平面重建模型替代耗时的几何优化提速1000倍,并结合无需训练的IP-Adapter模块提升纹理一致性,实现了高效且高质量的复杂3D资产生成。

摘要:

我们介绍了DreamCraft3D++,它是DreamCraft3D的扩展,能够高效生成高质量的复杂3D资产。DreamCraft3D++继承了DreamCraft3D的多阶段生成过程,但用前馈式多平面重建模型取代了耗时的几何雕刻优化,将处理速度提升了1000倍。在纹理优化方面,我们提出了无需训练的IP-Adapter模块,该模块以增强的多视图图像为条件,以增强纹理和几何一致性,提供了比DreamCraft3D的DreamBooth微调快4倍的替代方案。在多样化数据集上的实验表明,DreamCraft3D++能够生成具有复杂几何结构和逼真360°纹理的创意3D资产,在质量和速度上均优于最先进的图像到3D方法。完整实现将开源,为3D内容创作提供新的可能。

英文摘要:

We introduce DreamCraft3D++, an extension of DreamCraft3D that enables efficient high-quality generation of complex 3D assets. DreamCraft3D++ inherits the multi-stage generation process of DreamCraft3D, but replaces the time-consuming geometry sculpting optimization with a feed-forward multi-plane based reconstruction model, speeding up the process by 1000x. For texture refinement, we propose a training-free IP-Adapter module that is conditioned on the enhanced multi-view images to enhance texture and geometry consistency, providing a 4x faster alternative to DreamCraft3D's DreamBooth fine-tuning. Experiments on diverse datasets demonstrate DreamCraft3D++'s ability to generate creative 3D assets with intricate geometry and realistic 360° textures, outperforming state-of-the-art image-to-3D methods in quality and speed. The full implementation will be open-sourced to enable new possibilities in 3D content creation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑