arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2312.11459cs.CV
收藏

VolumeDiffusion:基于高效体编码器的灵活文本到3D生成

VolumeDiffusion: Flexible Text-to-3D Generation with Efficient Volumetric Encoder

Tsinghua University(清华大学);Microsoft Research(微软研究院)

Zhicong Tang, Shuyang Gu, Chunyu Wang, Ting Zhang, Jianmin Bao, Dong Chen, Baining Guo

AI总结:

本研究提出VolumeDiffusion文本到3D生成方法,通过轻量级网络从多视图图像获取特征体,结合3D U-Net扩散模型训练,在Objaverse数据集上实现了多样可控的3D生成效果。

摘要:

本文提出了一种面向文本到3D生成的开创性3D体编码器。为扩大扩散模型的训练数据规模,研究团队开发了一个轻量级网络,可从多视图图像中高效获取特征体。随后,这些3D体通过3D U-Net在扩散模型上训练,用于文本到3D生成。本研究进一步解决了物体描述不准确和高维特征体带来的挑战。所提模型在公开的Objaverse数据集上训练,在根据文本提示生成多样且可识别的样本方面展现出良好效果。值得注意的是,它能通过文本提示实现对物体部件特征的更精细控制,通过在单个物体中无缝融合多个概念来提升模型的创造力。本研究提出了一种高效、灵活且可扩展的表示方法,为3D生成的发展做出了重要贡献。

英文摘要:

This paper introduces a pioneering 3D volumetric encoder designed for text-to-3D generation. To scale up the training data for the diffusion model, a lightweight network is developed to efficiently acquire feature volumes from multi-view images. The 3D volumes are then trained on a diffusion model for text-to-3D generation using a 3D U-Net. This research further addresses the challenges of inaccurate object captions and high-dimensional feature volumes. The proposed model, trained on the public Objaverse dataset, demonstrates promising outcomes in producing diverse and recognizable samples from text prompts. Notably, it empowers finer control over object part characteristics through textual cues, fostering model creativity by seamlessly combining multiple concepts within a single object. This research significantly contributes to the progress of 3D generation by introducing an efficient, flexible, and scalable representation methodology.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑