arXiv 2312.021552024-04-16cs.CV

GPS-Gaussian：用于实时人体新视角合成的可泛化像素级 3D Gaussian Splatting

GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesis

Harbin Institute of Technology（哈尔滨工业大学）；Tsinghua University（清华大学）；Peng Cheng Laboratory（鹏城实验室）

Shunyuan Zheng, Boyao Zhou, Ruizhi Shao, Boning Liu, Shengping Zhang, Liqiang Nie, Yebin Liu

备注：: Accepted by CVPR 2024 (Highlight). Project page: https://shunyuanzheng.github.io/GPS-Gaussian

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出 GPS-Gaussian，通过源视角 Gaussian 参数图与深度估计联合回归，实现稀疏视角下无需逐人优化的实时 2K 人体新视角合成。

摘要：

我们提出了一种名为 GPS-Gaussian 的新方法，用于实时合成人物角色的新视角。该方法能够在稀疏视角相机设置下实现 2K 分辨率渲染。不同于原始 Gaussian Splatting 或神经隐式渲染方法需要针对每个对象进行优化，我们引入了在源视角上定义的 Gaussian 参数图，并直接回归 Gaussian Splatting 属性，从而无需任何微调或优化即可即时完成新视角合成。为此，我们在大量人体扫描数据上训练 Gaussian 参数回归模块，并联合一个深度估计模块，将二维参数图提升到三维空间。所提出的框架完全可微；在多个数据集上的实验表明，本方法优于最先进方法，同时实现了极高的渲染速度。

英文摘要：

We present a new approach, termed GPS-Gaussian, for synthesizing novel views of a character in a real-time manner. The proposed method enables 2K-resolution rendering under a sparse-view camera setting. Unlike the original Gaussian Splatting or neural implicit rendering methods that necessitate per-subject optimizations, we introduce Gaussian parameter maps defined on the source views and regress directly Gaussian Splatting properties for instant novel view synthesis without any fine-tuning or optimization. To this end, we train our Gaussian parameter regression module on a large amount of human scan data, jointly with a depth estimation module to lift 2D parameter maps to 3D space. The proposed framework is fully differentiable and experiments on several datasets demonstrate that our method outperforms state-of-the-art methods while achieving an exceeding rendering speed.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。