AGG:用于单图像到3D的摊销生成3D高斯
AGG: Amortized Generative 3D Gaussians for Single Image to 3D
AI总结:
提出摊销生成3D高斯框架AGG,通过混合表示和级联超分辨率模块,实现从单张图像快速生成高质量3D高斯,无需逐实例优化,速度提升数个数量级。
摘要:
鉴于对自动3D内容创建流程日益增长的需求,人们已经研究了各种3D表示来从单张图像生成3D对象。由于其出色的渲染效率,基于3D Gaussian splatting的模型最近在3D重建和生成方面均表现出色。用于图像到3D生成的3D Gaussian splatting方法通常基于优化,需要许多计算成本高昂的分数蒸馏步骤。为了克服这些挑战,我们引入了一种摊销生成3D高斯框架(AGG),该框架能从单张图像即时生成3D高斯,消除了逐实例优化的需要。利用中间混合表示,AGG将3D高斯位置和其他外观属性的生成分解以进行联合优化。此外,我们提出了一个级联流程,首先生成3D数据的粗略表示,随后使用3D高斯超分辨率模块对其进行上采样。我们的方法与现有的基于优化的3D高斯框架以及利用其他3D表示的基于采样的流程进行了对比评估,其中AGG在定性和定量上均展现出具有竞争力的生成能力,同时速度快了几个数量级。
英文摘要:
Given the growing need for automatic 3D content creation pipelines, various 3D representations have been studied to generate 3D objects from a single image. Due to its superior rendering efficiency, 3D Gaussian splatting-based models have recently excelled in both 3D reconstruction and generation. 3D Gaussian splatting approaches for image to 3D generation are often optimization-based, requiring many computationally expensive score-distillation steps. To overcome these challenges, we introduce an Amortized Generative 3D Gaussian framework (AGG) that instantly produces 3D Gaussians from a single image, eliminating the need for per-instance optimization. Utilizing an intermediate hybrid representation, AGG decomposes the generation of 3D Gaussian locations and other appearance attributes for joint optimization. Moreover, we propose a cascaded pipeline that first generates a coarse representation of the 3D data and later upsamples it with a 3D Gaussian super-resolution module. Our method is evaluated against existing optimization-based 3D Gaussian frameworks and sampling-based pipelines utilizing other 3D representations, where AGG showcases competitive generation abilities both qualitatively and quantitatively while being several orders of magnitude faster. Project page: https://ir1d.github.io/AGG/
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。