arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.12019cs.CV
收藏

LN3DIFF++: 可扩展的潜在神经场扩散用于快速3D生成

LN3DIFF++: Scalable Latent Neural Fields Diffusion for Speedy 3D Generation

Yushi Lan, Fangzhou Hong, Shangchen Zhou, Shuai Yang, Xuyi Meng, Yongwei Chen, Zhaoyang Lyu, Bo Dai, Xingang Pan, Chen Change Loy

AI总结:

LN3DIFF++通过3D感知架构和变分自编码器实现快速高质量的3D生成,优于现有方法在推理速度和生成质量上。

摘要:

神经渲染领域在生成模型和可微渲染技术的进步下取得了显著进展。尽管2D扩散已取得成功,但统一的3D扩散管道仍存缺口。本文介绍了一种名为LN3Diff++的新框架,以解决这一缺口并实现快速、高质量和通用的条件3D生成。我们的方法利用一种3D感知架构和变分自编码器(VAE)将输入图像编码为结构化、紧凑和3D的潜在空间。该潜在空间通过基于Transformer的解码器解码为高容量的3D神经场。通过在该3D感知潜在空间上训练扩散模型,我们的方法在ShapeNet上实现了3D生成的最先进性能,并在单目3D重建和各种数据集上的条件3D生成中表现出优越性能。此外,它在推理速度方面优于现有3D扩散方法,无需每个实例的优化。我们提出的方法在3D生成建模方面具有重大进展,并在3D视觉和图形任务中具有广泛应用前景。

英文摘要:

The field of neural rendering has witnessed significant progress with advancements in generative models and differentiable rendering techniques. Though 2D diffusion has achieved success, a unified 3D diffusion pipeline remains unsettled. This paper introduces a novel framework called LN3Diff++ to address this gap and enable fast, high-quality, and generic conditional 3D generation. Our approach harnesses a 3D-aware architecture and variational autoencoder (VAE) to encode the input image into a structured, compact, and 3D latent space. The latent is decoded by a transformer-based decoder into a high-capacity 3D neural field. Through training a diffusion model on this 3D-aware latent space, our method achieves state-of-the-art performance on ShapeNet for 3D generation and demonstrates superior performance in monocular 3D reconstruction and conditional 3D generation across various datasets. Moreover, it surpasses existing 3D diffusion methods in terms of inference speed, requiring no per-instance optimization. Our proposed LN3Diff presents a significant advancement in 3D generative modeling and holds promise for various applications in 3D vision and graphics tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑