arXiv 2410.110802024-10-14cs.CVcs.AIcs.GR

使用深度感知3D高斯泼溅的少样本新视角合成

Few-shot Novel View Synthesis using Depth Aware 3D Gaussian Splatting

University of California Santa Cruz（加州大学圣克鲁兹分校）

Raja Kumar, Vanshika Vats

备注：: Presented in ECCV 2024 workshop S3DSGR

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出一种用于少样本新视角合成的深度感知3D高斯泼溅方法，利用单目深度先验、尺度不变深度损失、低阶球谐函数建模并保留所有泼溅，显著提升了少视角下的重建质量与渲染效果。

摘要：

3D高斯泼溅通过实现更低的计算成本和实时高质量渲染，在新视角合成中超越了神经辐射场方法。尽管它能利用大量输入视角产生高质量渲染，但当仅有少量视角可用时，其性能会显著下降。在本研究中，我们通过提出一种用于少样本新视角合成的深度感知高斯泼溅方法来解决这一问题。我们使用单目深度预测作为先验，并结合尺度不变深度损失，在仅有少量输入视角的情况下约束3D形状。我们还使用低阶球谐函数对颜色进行建模以避免过拟合。此外，我们观察到，像原始工作中那样定期移除具有较低不透明度的泼溅会导致非常稀疏的点云，从而导致渲染质量下降。为了缓解这一问题，我们保留了所有泼溅，在少视角设置下实现了更好的重建。实验结果表明，我们的方法优于传统的3D高斯泼溅方法，在峰值信噪比上提升了10.5%，在结构相似性指数上提升了6%，在感知相似性上提升了14.1%，从而验证了我们方法的有效性。代码将发布于：https://github.com/raja-kumar/depth-aware-3DGS

英文摘要：

3D Gaussian splatting has surpassed neural radiance field methods in novel view synthesis by achieving lower computational costs and real-time high-quality rendering. Although it produces a high-quality rendering with a lot of input views, its performance drops significantly when only a few views are available. In this work, we address this by proposing a depth-aware Gaussian splatting method for few-shot novel view synthesis. We use monocular depth prediction as a prior, along with a scale-invariant depth loss, to constrain the 3D shape under just a few input views. We also model color using lower-order spherical harmonics to avoid overfitting. Further, we observe that removing splats with lower opacity periodically, as performed in the original work, leads to a very sparse point cloud and, hence, a lower-quality rendering. To mitigate this, we retain all the splats, leading to a better reconstruction in a few view settings. Experimental results show that our method outperforms the traditional 3D Gaussian splatting methods by achieving improvements of 10.5% in peak signal-to-noise ratio, 6% in structural similarity index, and 14.1% in perceptual similarity, thereby validating the effectiveness of our approach. The code will be made available at: https://github.com/raja-kumar/depth-aware-3DGS

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文