arXiv 2501.047822025-01-08cs.CV

GaussianVideo：通过分层高斯泼溅实现高效视频表示

GaussianVideo: Efficient Video Representation via Hierarchical Gaussian Splatting

Koç University（科克大学）；Adobe Research（奥多比研究院）；Hacettepe University（哈杰泰佩大学）

Andrew Bond, Jui-Hsien Wang, Long Mai, Erkut Erdem, Aykut Erdem

备注：: 10 pages, 10 figures

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出GaussianVideo，融合3D Gaussian splatting、Neural ODEs相机轨迹建模与时空分层学习，实现低内存、快速收敛且时间一致的高质量视频渲染，并在多类视频数据集上达到SOTA。

摘要：

用于动态视频场景的高效神经表示对于从视频压缩到交互式仿真等应用都至关重要。然而，现有方法往往面临内存占用高、训练时间长以及时间一致性方面的挑战。为解决这些问题，我们提出一种新型神经视频表示，将3D Gaussian splatting（三维高斯泼溅）与连续相机运动建模相结合。通过利用Neural ODEs，我们的方法学习平滑的相机轨迹，同时通过高斯维持显式的三维场景表示。此外，我们提出一种时空分层学习策略，逐步细化空间和时间特征，以提升重建质量并加快收敛。这种内存高效的方法以令人瞩目的速度实现高质量渲染。实验结果表明，我们的分层学习与稳健的相机运动建模相结合，能够捕捉具有强时间一致性的复杂动态场景，并在高运动和低运动场景下的多种视频数据集上取得最先进性能。

英文摘要：

Efficient neural representations for dynamic video scenes are critical for applications ranging from video compression to interactive simulations. Yet, existing methods often face challenges related to high memory usage, lengthy training times, and temporal consistency. To address these issues, we introduce a novel neural video representation that combines 3D Gaussian splatting with continuous camera motion modeling. By leveraging Neural ODEs, our approach learns smooth camera trajectories while maintaining an explicit 3D scene representation through Gaussians. Additionally, we introduce a spatiotemporal hierarchical learning strategy, progressively refining spatial and temporal features to enhance reconstruction quality and accelerate convergence. This memory-efficient approach achieves high-quality rendering at impressive speeds. Experimental results show that our hierarchical learning, combined with robust camera motion modeling, captures complex dynamic scenes with strong temporal consistency, achieving state-of-the-art performance across diverse video datasets in both high- and low-motion scenarios.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文