利用流溅射从单目视频学习高效4D高斯表示
Learning Efficient 4D Gaussian Representations from Monocular Videos with Flow Splatting
AI总结：
提出Flow Splatting方法，通过构建速度场并扩展溅射技术渲染光流，从单目视频中高效学习4D高斯表示，实现高质量动态场景重建。
摘要：
从单目视频重建动态3D场景因场景复杂性和时间动态而具有挑战性。随着3D高斯溅射在新视角合成中的进展，现有方法通过变形场、轨迹或时空4D体积将3D高斯扩展到4D域以建模场景元素变形。然而，这些方法在每帧重建4D体积时存在训练时间长、渲染速度慢或内存消耗高的问题，未能充分利用密集动态信息。为解决此问题，我们提出Flow Splatting，该方法构建速度场并使传统溅射技术能够从速度场渲染光流，以监督从单目视频学习动态过程。具体而言，我们扩展4D体积，使用随时间变化的均值和协方差来表示复杂动态。然后，基于该表示自然地构建并近似速度场。传统体渲染技术支持渲染颜色场，我们扩展体渲染策略，通过考虑相机运动的影响来溅射速度场。我们在各种基准上进行了实验，以证明我们方法的效率和有效性。与最先进方法相比，我们的模型在更少时间消耗和更高渲染速度下实现了更好的图像质量。
英文摘要：
Reconstructing dynamic 3D scenes from monocular videos is challenging due to scene complexity and temporal dynamics. With the advancement of 3D Gaussian Splatting in novel view synthesis, existing methods extend 3D Gaussians to 4D domain with deformation fields, trajectories or spatiotemporal 4D volumes to model scene element deformation. However, these methods suffer from long training time, low rendering speed or high memory consumption for per-frame reconstruction of 4D volumes, without fully exploiting dense dynamic information. To address this issue, we propose Flow Splatting, which constructs the velocity field and enables the conventional splatting technique to render optical flow from the velocity field to supervise dynamics learning process from monocular videos. Specifically, we extend 4D volumes with time varying means and covariance to represent complex dynamics. Then, we construct and approximate the velocity field naturally based on this representations. While conventional volume rendering techniques support to render color fields, we extend the volume rendering strategy to splat the velocity field by considering the influence of camera motions. We conduct experiments on various benchmarks to demonstrate the efficiency and effectiveness of our method. Compared to the state-of-the-art methods, our model achieves better image quality with less time consumption and higher rendering speed.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
