GPS-Gaussian+：用于从稀疏视角实时人-场景渲染的可泛化像素级 3D Gaussian Splatting
GPS-Gaussian+: Generalizable Pixel-wise 3D Gaussian Splatting for Real-Time Human-Scene Rendering from Sparse Views
AI总结：
提出可泛化的像素级 3D Gaussian Splatting 框架，通过回归 Gaussian 参数图并结合深度估计与对极注意力机制，在稀疏视角下实现无需优化的实时高分辨率人-场景渲染。
摘要：
可微渲染技术近期在角色的自由视点视频合成中展现出可喜成果。然而，此类方法（无论是 Gaussian Splatting 还是神经隐式渲染）通常需要针对每个主体进行优化，无法满足交互式应用中实时渲染的要求。我们提出了一种可泛化的 Gaussian Splatting 方法，用于稀疏视角相机设置下的高分辨率图像渲染。为此，我们引入了定义在源视图上的 Gaussian 参数图，并直接回归 Gaussian 属性，以实现无需任何微调或优化的即时新视图合成。我们在仅包含人体或人-场景的数据上训练 Gaussian 参数回归模块，并与深度估计模块联合，将 2D 参数图提升至 3D 空间。所提出的框架在同时具有深度和渲染监督，或仅有渲染监督时均完全可微。我们进一步引入了正则化项和对极注意力机制，以保持两个源视图之间的几何一致性，尤其是在忽略深度监督时。在多个数据集上的实验表明，我们的方法在实现超快渲染速度的同时，优于现有最先进的方法。
英文摘要：
Differentiable rendering techniques have recently shown promising results for free-viewpoint video synthesis of characters. However, such methods, either Gaussian Splatting or neural implicit rendering, typically necessitate per-subject optimization which does not meet the requirement of real-time rendering in an interactive application. We propose a generalizable Gaussian Splatting approach for high-resolution image rendering under a sparse-view camera setting. To this end, we introduce Gaussian parameter maps defined on the source views and directly regress Gaussian properties for instant novel view synthesis without any fine-tuning or optimization. We train our Gaussian parameter regression module on human-only data or human-scene data, jointly with a depth estimation module to lift 2D parameter maps to 3D space. The proposed framework is fully differentiable with both depth and rendering supervision or with only rendering supervision. We further introduce a regularization term and an epipolar attention mechanism to preserve geometry consistency between two source views, especially when neglecting depth supervision. Experiments on several datasets demonstrate that our method outperforms state-of-the-art methods while achieving an exceeding rendering speed.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
