GaussianDreamerPro:从文本生成高质量可操作的3D高斯模型
GaussianDreamerPro: Text to Manipulable 3D Gaussians with Highly Enhanced Quality
AI总结:
提出GaussianDreamerPro框架,通过将3D高斯绑定到演化几何体上,从文本生成高质量、可操作(如动画、合成)的3D资产,显著提升细节与质量。
摘要:
近年来,3D高斯泼溅(3D-GS)在重建和渲染真实场景方面取得了巨大成功。为了将这种高渲染质量迁移到生成任务中,一系列研究工作尝试从文本生成3D高斯资产。然而,生成的资产在质量上并未达到重建任务中的水平。我们观察到,由于生成过程可能引发不确定性,高斯往往不受控制地增长。为了大幅提升生成质量,我们提出了一个名为GaussianDreamerPro的新框架。其核心思想是将高斯绑定到合理的几何体上,该几何体在整个生成过程中不断演化。在我们框架的不同阶段,几何和外观都能逐步得到丰富。最终的输出资产由绑定到网格上的3D高斯构成,与以往方法相比,展现出显著增强的细节和质量。值得注意的是,生成的资产还可以无缝集成到下游操作流程中,例如动画、合成和模拟等,极大地促进了其在广泛应用中的潜力。演示可在 https://taoranyi.com/gaussiandreamerpro/ 获取。
英文摘要:
Recently, 3D Gaussian splatting (3D-GS) has achieved great success in reconstructing and rendering real-world scenes. To transfer the high rendering quality to generation tasks, a series of research works attempt to generate 3D-Gaussian assets from text. However, the generated assets have not achieved the same quality as those in reconstruction tasks. We observe that Gaussians tend to grow without control as the generation process may cause indeterminacy. Aiming at highly enhancing the generation quality, we propose a novel framework named GaussianDreamerPro. The main idea is to bind Gaussians to reasonable geometry, which evolves over the whole generation process. Along different stages of our framework, both the geometry and appearance can be enriched progressively. The final output asset is constructed with 3D Gaussians bound to mesh, which shows significantly enhanced details and quality compared with previous methods. Notably, the generated asset can also be seamlessly integrated into downstream manipulation pipelines, e.g. animation, composition, and simulation etc., greatly promoting its potential in wide applications. Demos are available at https://taoranyi.com/gaussiandreamerpro/.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。