HiFi4G：通过紧凑高斯泼溅实现高保真人体表演渲染
HiFi4G: High-Fidelity Human Performance Rendering via Compact Gaussian Splatting
AI总结：
本文提出 HiFi4G，结合 3D Gaussian 与非刚性跟踪，通过双图机制、4D Gaussian 优化及残差补偿压缩，实现高保真、低存储的人体表演渲染。
摘要：
近来，照片级真实感人体建模与渲染取得了巨大进展。然而，高效渲染逼真的人体表演并将其集成到光栅化管线中仍具挑战。本文提出 HiFi4G，这是一种显式且紧凑的基于高斯的方法，用于从密集影像中渲染高保真人体表演。我们的核心直觉是将 3D Gaussian 表示与非刚性跟踪相结合，获得紧凑且利于压缩的表示。我们首先提出双图机制以获取运动先验：使用粗粒度变形图进行有效初始化，并使用细粒度 Gaussian 图施加后续约束。随后，我们采用带有自适应时空正则项的 4D Gaussian 优化方案，以有效平衡非刚性先验与 Gaussian 更新。我们还提出一种带残差补偿的配套压缩方案，用于在多种平台上提供沉浸式体验。该方案实现了约 25 倍的显著压缩率，每帧存储量不足 2MB。大量实验证明了本方法的有效性，其在优化速度、渲染质量和存储开销方面均显著优于现有方法。
英文摘要：
We have recently seen tremendous progress in photo-real human modeling and rendering. Yet, efficiently rendering realistic human performance and integrating it into the rasterization pipeline remains challenging. In this paper, we present HiFi4G, an explicit and compact Gaussian-based approach for high-fidelity human performance rendering from dense footage. Our core intuition is to marry the 3D Gaussian representation with non-rigid tracking, achieving a compact and compression-friendly representation. We first propose a dual-graph mechanism to obtain motion priors, with a coarse deformation graph for effective initialization and a fine-grained Gaussian graph to enforce subsequent constraints. Then, we utilize a 4D Gaussian optimization scheme with adaptive spatial-temporal regularizers to effectively balance the non-rigid prior and Gaussian updating. We also present a companion compression scheme with residual compensation for immersive experiences on various platforms. It achieves a substantial compression rate of approximately 25 times, with less than 2MB of storage per frame. Extensive experiments demonstrate the effectiveness of our approach, which significantly outperforms existing approaches in terms of optimization speed, rendering quality, and storage overhead.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
