arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.05054cs.CV
收藏

LGM:用于高分辨率3D内容生成的大型多视图高斯模型

LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation

National Key Lab of General AI, Peking University(北京大学通用人工智能重点实验室);S-Lab, Nanyang Technological University(南洋理工大学S-Lab实验室);Shanghai AI Lab(上海人工智能实验室)

Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, Ziwei Liu

AI总结:

本文提出LGM框架,利用多视图高斯特征和非对称U-Net,从文本或单视图图像快速生成高分辨率3D模型,5秒内完成,分辨率达512。

摘要:

3D内容创建在质量和速度方面都取得了显著进展。尽管当前的馈送式模型可以在几秒钟内生成3D对象,但其分辨率受到训练期间密集计算需求的限制。在本文中,我们引入了大型多视图高斯模型(LGM),这是一个新颖的框架,旨在从文本提示或单视图图像生成高分辨率3D模型。我们的关键见解有两点:1)3D表示:我们提出多视图高斯特征作为一种高效且强大的表示,可以融合在一起进行可微渲染。2)3D主干:我们提出了一种非对称U-Net作为高吞吐量的主干,处理多视图图像,这些图像可以通过利用多视图扩散模型从文本或单视图图像输入生成。大量实验证明了我们方法的高保真度和效率。值得注意的是,我们保持了在5秒内生成3D对象的快速速度,同时将训练分辨率提高到512,从而实现了高分辨率3D内容生成。

英文摘要:

3D content creation has achieved significant progress in terms of both quality and speed. Although current feed-forward models can produce 3D objects in seconds, their resolution is constrained by the intensive computation required during training. In this paper, we introduce Large Multi-View Gaussian Model (LGM), a novel framework designed to generate high-resolution 3D models from text prompts or single-view images. Our key insights are two-fold: 1) 3D Representation: We propose multi-view Gaussian features as an efficient yet powerful representation, which can then be fused together for differentiable rendering. 2) 3D Backbone: We present an asymmetric U-Net as a high-throughput backbone operating on multi-view images, which can be produced from text or single-view image input by leveraging multi-view diffusion models. Extensive experiments demonstrate the high fidelity and efficiency of our approach. Notably, we maintain the fast speed to generate 3D objects within 5 seconds while boosting the training resolution to 512, thereby achieving high-resolution 3D content generation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑