GaussianFlow：为4D内容创建泼溅高斯动力学
GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creation
AI总结：
针对4D高斯场创建约束不足、高斯运动直接监督缺失的问题，提出连接3D高斯动力学与帧间像素速度的高斯流概念，通过可微的动力学泼溅实现光流动态监督，解决颜色漂移问题，在4D生成与新视角合成任务上达SOTA。
摘要：
从图像或视频创建高斯泼溅（Gaussian Splatting）的4D场是一项具有挑战性的任务，因为其约束不足。虽然优化过程可以从输入视频中获取光度参考，或通过生成模型进行调节，但对高斯运动的直接监督仍未得到充分研究。本文提出了一个新概念——高斯流（Gaussian flow），它将3D高斯的动力学与连续帧之间的像素速度关联起来。通过将高斯动力学泼溅到图像空间中，可以高效地获得高斯流。这一可微过程实现了来自光流的直接动力学监督。我们的方法极大地助力了基于高斯泼溅的4D动态内容生成和4D新视角合成，尤其适用于现有方法难以处理的富含运动的内容。4D生成中常见的颜色漂移问题也通过改进的高斯动力学得到了解决。大量实验中展现出的优异视觉质量证明了我们方法的有效性。定量和定性评估表明，我们的方法在4D生成和4D新视角合成两项任务上均取得了最先进的结果。项目页面：https://zerg-overmind.github.io/GaussianFlow.github.io/
英文摘要：
Creating 4D fields of Gaussian Splatting from images or videos is a challenging task due to its under-constrained nature. While the optimization can draw photometric reference from the input videos or be regulated by generative models, directly supervising Gaussian motions remains underexplored. In this paper, we introduce a novel concept, Gaussian flow, which connects the dynamics of 3D Gaussians and pixel velocities between consecutive frames. The Gaussian flow can be efficiently obtained by splatting Gaussian dynamics into the image space. This differentiable process enables direct dynamic supervision from optical flow. Our method significantly benefits 4D dynamic content generation and 4D novel view synthesis with Gaussian Splatting, especially for contents with rich motions that are hard to be handled by existing methods. The common color drifting issue that happens in 4D generation is also resolved with improved Guassian dynamics. Superior visual quality on extensive experiments demonstrates our method's effectiveness. Quantitative and qualitative evaluations show that our method achieves state-of-the-art results on both tasks of 4D generation and 4D novel view synthesis. Project page: https://zerg-overmind.github.io/GaussianFlow.github.io/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
