具有不确定性感知正则化的野外4D高斯泼溅
4D Gaussian Splatting in the Wild with Uncertainty-Aware Regularization
AI总结：
提出一种基于不确定性感知正则化的4D高斯泼溅算法，通过扩散模型先验和动态区域致密化方法解决单目视频动态场景重建的过拟合与初始化问题，提升了新视角合成与重建质量。
摘要：
动态场景的新视角合成在包括增强现实和虚拟现实的各种应用中变得日益重要。我们提出了一种新颖的4D Gaussian Splatting (4DGS)算法，用于处理随意录制的单目视频中的动态场景。为了克服现有工作在这些真实世界视频上的过拟合问题，我们引入了一种不确定性感知正则化方法，该方法能够识别观测次数较少的不确定区域，并基于扩散模型和深度平滑性在这些区域上选择性地施加额外先验。这种方法既提升了新视角合成的性能，也改善了训练图像重建的质量。我们还发现了4DGS在快速运动的动态区域中的初始化问题，在这些区域中，Structure from Motion (SfM)算法无法提供可靠的3D地标。为了在这些区域初始化高斯基元，我们提出了一种使用估计深度图和场景流的动态区域致密化方法。我们的实验表明，所提出的方法提升了从手持单目相机拍摄的视频中进行4DGS重建的性能，并且在少样本静态场景重建中也展现出了令人满意的结果。
英文摘要：
Novel view synthesis of dynamic scenes is becoming important in various applications, including augmented and virtual reality. We propose a novel 4D Gaussian Splatting (4DGS) algorithm for dynamic scenes from casually recorded monocular videos. To overcome the overfitting problem of existing work for these real-world videos, we introduce an uncertainty-aware regularization that identifies uncertain regions with few observations and selectively imposes additional priors based on diffusion models and depth smoothness on such regions. This approach improves both the performance of novel view synthesis and the quality of training image reconstruction. We also identify the initialization problem of 4DGS in fast-moving dynamic regions, where the Structure from Motion (SfM) algorithm fails to provide reliable 3D landmarks. To initialize Gaussian primitives in such regions, we present a dynamic region densification method using the estimated depth maps and scene flow. Our experiments show that the proposed method improves the performance of 4DGS reconstruction from a video captured by a handheld monocular camera and also exhibits promising results in few-shot static scene reconstruction.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
