重新审视前馈 3D Gaussian Splatting 的深度表示
Revisiting Depth Representations for Feed-Forward 3D Gaussian Splatting
- Zhejiang University, China（浙江大学）
- Monash University, Australia（墨尔本大学）
- MBZUAI
- GigaAI（极佳科技）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文针对前馈 3DGS 中深度边界导致点云稀疏、渲染退化的问题，提出基于预训练 transformer pointmap 的 PM-Loss 正则化，以增强边界几何平滑并跨架构提升渲染质量。
AI中文摘要：
深度图被广泛用于前馈 3D Gaussian Splatting（3DGS）流程中，通过将其反投影为三维点云来进行新视角合成。该方法具有训练高效、可利用已知相机位姿以及几何估计准确等优势。然而，物体边界处的深度不连续性往往会导致点云碎片化或稀疏，从而降低渲染质量——这是基于深度的表示众所周知的局限。为解决这一问题，我们提出 PM-Loss，这是一种基于预训练 transformer 所预测 pointmap 的新型正则化损失。尽管 pointmap 本身可能不如深度图准确，但它能有效增强几何平滑性，尤其是在物体边界附近。借助改进后的深度图，我们的方法在多种架构和场景中显著提升了前馈 3DGS，持续带来更好的渲染结果。项目页面：https://aim-uofa.github.io/PMLoss
英文摘要：
Depth maps are widely used in feed-forward 3D Gaussian Splatting (3DGS) pipelines by unprojecting them into 3D point clouds for novel view synthesis. This approach offers advantages such as efficient training, the use of known camera poses, and accurate geometry estimation. However, depth discontinuities at object boundaries often lead to fragmented or sparse point clouds, degrading rendering quality -- a well-known limitation of depth-based representations. To tackle this issue, we introduce PM-Loss, a novel regularization loss based on a pointmap predicted by a pre-trained transformer. Although the pointmap itself may be less accurate than the depth map, it effectively enforces geometric smoothness, especially around object boundaries. With the improved depth map, our method significantly improves the feed-forward 3DGS across various architectures and scenes, delivering consistently better rendering results. Our project page: https://aim-uofa.github.io/PMLoss
