arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2503.24382cs.CV

Free360:基于分层高斯泼溅的从极稀疏且无位姿视角进行无界360度视图合成

Free360: Layered Gaussian Splatting for Unbounded 360-Degree View Synthesis from Extremely Sparse and Unposed Views

State Key Lab of CAD&CG, Zhejiang University(浙江大学CAD&CG国家重点实验室);ETH Zürich(苏黎世联邦理工学院);University of Tübingen, Tübingen AI Center(蒂宾根大学蒂宾根人工智能中心)

Chong Bao, Xiyu Zhang, Zehao Yu, Jiale Shi, Guofeng Zhang, Songyou Peng, Zhaopeng Cui

AI总结:

提出一种分层高斯泼溅神经渲染框架,通过密集立体重建、特定层引导优化及重建与生成的迭代融合,解决了无界360度场景中极稀疏无位姿视角的3D重建与新视角合成问题。

摘要:

神经渲染在具有密集输入视角和准确位姿的高质量3D神经重建与新视角合成中展现了显著的成功。然而,将其应用于无界360度场景中极稀疏且无位姿的视角仍是一个具有挑战性的问题。在本文中,我们提出了一种新颖的神经渲染框架,以在无界360度场景中完成无位姿和极稀疏视角的3D重建。为了解决稀疏输入视角下无界场景固有的空间歧义,我们提出了一种基于分层高斯的表示方法,以不同的空间层对场景进行有效建模。通过采用密集立体重建模型来恢复粗略几何,我们引入了特定层的引导优化,以细化噪声并填补重建中的遮挡区域。此外,我们提出了重建与生成的迭代融合以及一种不确定性感知的训练方法,以促进这两个过程之间的相互条件约束和增强。综合实验表明,我们的方法在渲染质量和表面重建精度方面优于现有的最先进方法。项目页面:https://zju3dv.github.io/free360/

英文摘要:

Neural rendering has demonstrated remarkable success in high-quality 3D neural reconstruction and novel view synthesis with dense input views and accurate poses. However, applying it to extremely sparse, unposed views in unbounded 360° scenes remains a challenging problem. In this paper, we propose a novel neural rendering framework to accomplish the unposed and extremely sparse-view 3D reconstruction in unbounded 360° scenes. To resolve the spatial ambiguity inherent in unbounded scenes with sparse input views, we propose a layered Gaussian-based representation to effectively model the scene with distinct spatial layers. By employing a dense stereo reconstruction model to recover coarse geometry, we introduce a layer-specific bootstrap optimization to refine the noise and fill occluded regions in the reconstruction. Furthermore, we propose an iterative fusion of reconstruction and generation alongside an uncertainty-aware training approach to facilitate mutual conditioning and enhancement between these two processes. Comprehensive experiments show that our approach outperforms existing state-of-the-art methods in terms of rendering quality and surface reconstruction accuracy. Project page: https://zju3dv.github.io/free360/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑