arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2511.07142cs.CV
收藏

ProcGen3D:学习用于图像到三维重建的神经程序化图表示

ProcGen3D: Learning Neural Procedural Graph Representations for Image-to-3D Reconstruction

Technical University of Munich(慕尼黑技术大学);Tsinghua University(清华大学)

Xinyi Zhang, Daoyi Gao, Naiqi Li, Angela Dai

AI总结:

ProcGen3D提出基于程序化图表示和transformer先验的图像到三维重建方法,并引入MCTS引导采样提升图像对齐,在多种对象上优于现有生成式三维方法。

摘要:

我们提出ProcGen3D,一种通过生成三维对象的程序化图抽象来进行三维内容创建的新方法,这些抽象随后可被解码为丰富、复杂的三维资产。受生产级三维应用中普遍使用程序化生成器的启发,我们为三维资产提出一种序列化的、基于图的程序化图表示。我们利用该表示来学习逼近用于基于图像的三维重建的程序化生成器的解空间。我们采用基于边的分词来编码程序化图,并训练一个transformer先验模型,以输入RGB图像为条件预测下一个token。关键的是,为了使生成输出与输入图像更好地对齐,我们在生成过程中引入蒙特卡洛树搜索(MCTS)引导采样,引导输出程序化图朝向更忠实于图像的重建结果。我们的方法适用于各种可通过程序化生成器合成的对象。在仙人掌、树木和桥梁上的大量实验表明,我们的神经程序化图生成优于最先进的生成式三维方法和领域特定的建模技术。此外,尽管仅在合成数据上训练,该方法仍能在真实世界输入图像上实现更好的泛化。

英文摘要:

We introduce ProcGen3D, a new approach for 3D content creation by generating procedural graph abstractions of 3D objects, which can then be decoded into rich, complex 3D assets. Inspired by the prevalent use of procedural generators in production 3D applications, we propose a sequentialized, graph-based procedural graph representation for 3D assets. We use this to learn to approximate the landscape of a procedural generator for image-based 3D reconstruction. We employ edge-based tokenization to encode the procedural graphs, and train a transformer prior to predict the next token conditioned on an input RGB image. Crucially, to enable better alignment of our generated outputs to an input image, we incorporate Monte Carlo Tree Search (MCTS) guided sampling into our generation process, steering output procedural graphs towards more image-faithful reconstructions. Our approach is applicable across a variety of objects that can be synthesized with procedural generators. Extensive experiments on cacti, trees, and bridges show that our neural procedural graph generation outperforms both state-of-the-art generative 3D methods and domain-specific modeling techniques. Furthermore, this enables improved generalization on real-world input images, despite training only on synthetic data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑