arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.15643cs.CVcs.GR
收藏

驯服3DGS:资源受限下的高质量辐射场

Taming 3DGS: High-Quality Radiance Fields with Limited Resources

Carnegie Mellon University(卡内基梅隆大学);Graz University of Technology(格拉茨工业大学);International Institute of Information Technology, Hyderabad(海得拉巴国际信息技术学院)

Saswat Subhajyoti Mallick, Rahul Goel, Bernhard Kerbl, Francisco Vicente Carrasco, Markus Steinberger, Fernando De La Torre

AI总结:

针对3DGS资源需求高、模型大小不定等问题,提出有引导的构造式致密化、数值等价的快速训练方案等改进,实现了资源受限下的高质量新视角合成。

摘要:

3D Gaussian Splatting(3DGS,三维高斯泼溅)凭借快速、可解释且高保真的渲染能力革新了新视角合成领域。然而,其资源需求限制了可用性:尤其在受限设备上,训练性能会快速下降,且常因模型内存消耗过高而无法完成训练。该方法收敛时的高斯数量不确定——其中许多是冗余的——导致渲染速度不必要地变慢,也无法应用于要求固定大小输入的下游任务。为解决这些问题,我们应对了在有限预算下训练和渲染3DGS模型的挑战。我们采用一种有引导的纯构造式致密化过程,将致密化导向能提升重建质量的高斯。模型大小以可控方式持续增长,直至达到精确预算,这一过程使用基于分数的高斯致密化,并结合训练时先验来衡量高斯的贡献。我们进一步解决了训练速度障碍:在仔细分析3DGS原始流水线后,我们推导出了用于梯度计算和属性更新的更快、数值等价的解决方案,包括一种用于高效反向传播的替代并行化方案。我们还在合适的情况下提出了保质量的近似方法,以进一步缩短训练时间。综上,这些改进形成了一个鲁棒、可扩展的解决方案,具有更短的训练时间、更低的计算和内存需求,且质量很高。我们的评估表明,在预算受限的设置下,我们获得了与3DGS相当的质量指标,同时模型大小和训练时间均减少了4-5倍。在预算更充足的情况下,我们测得的质量超过了3DGS。这些进展为受限环境(如移动设备)中的新视角合成打开了大门。

英文摘要:

3D Gaussian Splatting (3DGS) has transformed novel-view synthesis with its fast, interpretable, and high-fidelity rendering. However, its resource requirements limit its usability. Especially on constrained devices, training performance degrades quickly and often cannot complete due to excessive memory consumption of the model. The method converges with an indefinite number of Gaussians -- many of them redundant -- making rendering unnecessarily slow and preventing its usage in downstream tasks that expect fixed-size inputs. To address these issues, we tackle the challenges of training and rendering 3DGS models on a budget. We use a guided, purely constructive densification process that steers densification toward Gaussians that raise the reconstruction quality. Model size continuously increases in a controlled manner towards an exact budget, using score-based densification of Gaussians with training-time priors that measure their contribution. We further address training speed obstacles: following a careful analysis of 3DGS' original pipeline, we derive faster, numerically equivalent solutions for gradient computation and attribute updates, including an alternative parallelization for efficient backpropagation. We also propose quality-preserving approximations where suitable to reduce training time even further. Taken together, these enhancements yield a robust, scalable solution with reduced training times, lower compute and memory requirements, and high quality. Our evaluation shows that in a budgeted setting, we obtain competitive quality metrics with 3DGS while achieving a 4--5x reduction in both model size and training time. With more generous budgets, our measured quality surpasses theirs. These advances open the door for novel-view synthesis in constrained environments, e.g., mobile devices.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑