单目动态高斯泼溅:快速、脆弱,且场景复杂度决定一切
Monocular Dynamic Gaussian Splatting: Fast, Brittle, and Scene Complexity Rules
AI总结:
本文系统整理并基准测试多种单目动态 Gaussian splatting 方法,通过现有数据集与受控合成数据集比较运动表示,揭示真实场景复杂度会掩盖方法差异且高速渲染伴随优化脆弱性。
摘要:
Gaussian splatting(高斯泼溅)方法正成为一种流行方法,用于将多视图图像数据转换为支持视图合成的场景表示。特别是,人们关注仅使用单目输入数据实现动态场景的视图合成——这是一个病态且具有挑战性的问题。该领域工作进展迅速,同时出现了多篇论文,均声称自己的方法效果最佳,但这些说法不可能同时成立。在本工作中,我们对多种基于 Gaussian splatting 的方法进行整理、基准测试和分析,提供了以往工作所缺乏的同口径比较。我们使用多个现有数据集,以及一个新的、具有启发性的合成数据集;该数据集旨在分离影响重建质量的因素。我们系统地将 Gaussian splatting 方法划分为特定的运动表示类型,并量化它们之间的差异如何影响性能。根据实验,我们发现它们的排名顺序在合成数据中是明确的,但真实世界数据的复杂度目前压倒了这些差异。此外,所有基于 Gaussian 的方法所具有的快速渲染速度,是以优化过程中的脆弱性为代价的。我们将实验总结为一系列发现,以帮助推动这一活跃问题设定取得进一步进展。
英文摘要:
Gaussian splatting methods are emerging as a popular approach for converting multi-view image data into scene representations that allow view synthesis. In particular, there is interest in enabling view synthesis for dynamic scenes using only monocular input data -- an ill-posed and challenging problem. The fast pace of work in this area has produced multiple simultaneous papers that claim to work best, which cannot all be true. In this work, we organize, benchmark, and analyze many Gaussian-splatting-based methods, providing apples-to-apples comparisons that prior works have lacked. We use multiple existing datasets and a new instructive synthetic dataset designed to isolate factors that affect reconstruction quality. We systematically categorize Gaussian splatting methods into specific motion representation types and quantify how their differences impact performance. Empirically, we find that their rank order is well-defined in synthetic data, but the complexity of real-world data currently overwhelms the differences. Furthermore, the fast rendering speed of all Gaussian-based methods comes at the cost of brittleness in optimization. We summarize our experiments into a list of findings that can help to further progress in this lively problem setting.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。