GaussianVideo：基于Gaussian Splatting的高效视频表示与压缩
GaussianVideo: Efficient Video Representation and Compression by Gaussian Splatting
AI总结：
针对视频隐式神经表示编解码慢、内存高的问题，提出基于可变形2D Gaussian Splatting的视频表示与压缩方法，利用时空编码与时间梯度，实现高达78.4%的显存降低及显著的训练与解码加速。
摘要：
视频隐式神经表示（NeRV）引入了一种新的视频表示和压缩范式，性能超越了传统编解码器。然而，随着模型规模的增长，缓慢的编解码速度和极高的内存消耗阻碍了其在实际中的应用。为了解决这些限制，我们提出了一种基于2D Gaussian Splatting的新型视频表示和压缩方法，以高效处理视频数据。我们提出的可变形2D Gaussian Splatting在每一帧动态调整2D Gaussians的变换，显著降低了内存开销。结合基于多平面的时空编码器和轻量级解码器，它能在给定时间步长的情况下预测初始化Gaussians的颜色、坐标和形状的变化。通过利用时间梯度，我们的模型以可忽略的成本有效捕获了时间冗余，显著提升了视频表示效率。该方法将GPU内存使用量降低了高达78.4%，并显著加快了视频处理速度，与最先进的NeRV方法相比，实现了5.5倍的训练加速和12.5倍的解码加速。
英文摘要：
Implicit Neural Representation for Videos (NeRV) has introduced a novel paradigm for video representation and compression, outperforming traditional codecs. As model size grows, however, slow encoding and decoding speed and high memory consumption hinder its application in practice. To address these limitations, we propose a new video representation and compression method based on 2D Gaussian Splatting to efficiently handle video data. Our proposed deformable 2D Gaussian Splatting dynamically adapts the transformation of 2D Gaussians at each frame, significantly reducing memory cost. Equipped with a multi-plane-based spatiotemporal encoder and a lightweight decoder, it predicts changes in color, coordinates, and shape of initialized Gaussians, given the time step. By leveraging temporal gradients, our model effectively captures temporal redundancy at negligible cost, significantly enhancing video representation efficiency. Our method reduces GPU memory usage by up to 78.4%, and significantly expedites video processing, achieving 5.5x faster training and 12.5x faster decoding compared to the state-of-the-art NeRV methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
