从散焦和运动模糊的单目视频中进行动态高斯泼溅
Dynamic Gaussian Splatting from Defocused and Motion-blurred Monocular Videos
AI总结：
本文提出统一框架，通过模糊预测网络估计逐像素可靠模糊核，结合动态高斯致密化策略，从散焦和运动模糊的单目视频中实现高质量动态高斯泼溅与新视角合成。
摘要：
本文提出了一个统一框架，能够从散焦和运动模糊的单目视频中实现高质量的动态高斯泼溅。由于散焦模糊和运动模糊的形成过程存在显著差异，现有方法通常仅针对其中一种进行设计，缺乏同时处理两者的能力。尽管两者可以联合建模为基于模糊核的卷积，但估计准确模糊核的固有困难极大地限制了该方向的进展。在本研究中，我们在此方向上更进一步。具体而言，我们提出使用一个模糊预测网络来估计每个像素的可靠模糊核，该网络利用与模糊相关的场景和相机信息，并受模糊感知稀疏性约束。此外，我们引入了动态高斯致密化策略，以缓解不完整区域高斯缺失的问题，并通过结合未见视角信息约束场景优化，从而提升新视角合成的性能。大量实验表明，我们的方法在从散焦和运动模糊的单目视频中生成逼真的新视角合成方面优于现有最先进方法。代码可在 https://github.com/hhhddddddd/dydeblur 获取。
英文摘要：
This paper presents a unified framework that allows high-quality dynamic Gaussian Splatting from both defocused and motion-blurred monocular videos. Due to the significant difference between the formation processes of defocus blur and motion blur, existing methods are tailored for either one of them, lacking the ability to simultaneously deal with both of them. Although the two can be jointly modeled as blur kernel-based convolution, the inherent difficulty in estimating accurate blur kernels greatly limits the progress in this direction. In this work, we go a step further towards this direction. Particularly, we propose to estimate per-pixel reliable blur kernels using a blur prediction network that exploits blur-related scene and camera information and is subject to a blur-aware sparsity constraint. Besides, we introduce a dynamic Gaussian densification strategy to mitigate the lack of Gaussians for incomplete regions, and boost the performance of novel view synthesis by incorporating unseen view information to constrain scene optimization. Extensive experiments show that our method outperforms the state-of-the-art methods in generating photorealistic novel view synthesis from defocused and motion-blurred monocular videos. Our code is available at https://github.com/hhhddddddd/dydeblur.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
