arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2207.09446cs.CVcs.AI
收藏

ShapeCrafter:一个递归的文本条件化3D形状生成模型

ShapeCrafter: A Recursive Text-Conditioned 3D Shape Generation Model

Brown University(布朗大学)

Rao Fu, Xiao Zhan, Yiwen Chen, Daniel Ritchie, Srinath Sridhar

AI总结:

本文提出ShapeCrafter,一种递归文本条件化3D形状生成神经网络,基于Text2Shape++数据集和向量量化深度隐式函数,支持根据逐步添加的文本短语演化生成高质量3D形状,并实现形状编辑与外推。

摘要:

我们提出了ShapeCrafter,一个用于递归的文本条件化3D形状生成的神经网络。现有的生成文本条件化3D形状的方法在单步中消耗整个文本提示来生成3D形状。然而,人类倾向于递归地描述形状——我们可能从一个初始描述开始,并基于中间结果逐步添加细节。为了捕捉这一递归过程,我们引入了一种方法来生成3D形状分布,该分布以初始短语为条件,随着更多短语的添加而逐渐演化。由于现有数据集不足以训练这种方法,我们提出了Text2Shape++,一个包含369K形状-文本对的大型数据集,支持递归形状生成。为了捕捉常用于细化形状描述的局部细节,我们建立在向量量化深度隐式函数的基础上,这些函数生成高质量形状的分布。结果表明,我们的方法能够生成与文本描述一致的形状,并且随着更多短语的添加,形状会逐渐演化。我们的方法支持形状编辑、外推,并且能够在创意设计的人机协作中实现新的应用。

英文摘要:

We present ShapeCrafter, a neural network for recursive text-conditioned 3D shape generation. Existing methods to generate text-conditioned 3D shapes consume an entire text prompt to generate a 3D shape in a single step. However, humans tend to describe shapes recursively-we may start with an initial description and progressively add details based on intermediate results. To capture this recursive process, we introduce a method to generate a 3D shape distribution, conditioned on an initial phrase, that gradually evolves as more phrases are added. Since existing datasets are insufficient for training this approach, we present Text2Shape++, a large dataset of 369K shape-text pairs that supports recursive shape generation. To capture local details that are often used to refine shape descriptions, we build on top of vector-quantized deep implicit functions that generate a distribution of high-quality shapes. Results show that our method can generate shapes consistent with text descriptions, and shapes evolve gradually as more phrases are added. Our method supports shape editing, extrapolation, and can enable new applications in human-machine collaboration for creative design.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑