新颖的视图合成技术迅速发展,从神经辐射场发展到3D高斯点云渲染(3DGS),后者提供了实时渲染和快速训练,且不牺牲视觉保真度。然而,3DGS依赖于准确的相机姿态和高质量点云初始化,这在稀疏视角场景中难以获得。尽管传统结构从运动(SfM)管道在这些设置中往往失效,现有基于学习的点估计替代方法通常需要可靠的参考视角,并且对姿态或深度误差敏感。在本工作中,我们提出了一种利用π³的鲁棒方法,这是一种无参考点云估计网络。我们整合了来自π³的密集初始化,并设计了一种正则化方案以减轻几何不准确性。具体而言,我们采用不确定性引导的深度监督、法线一致性损失和深度变形。实验结果表明,我们的方法在 Tanks and Temples、LLFF、DTU 和 MipNeRF360 数据集上实现了最先进的性能。
英文摘要:
Novel view synthesis has evolved rapidly, advancing from Neural Radiance Fields to 3D Gaussian Splatting (3DGS), which offers real-time rendering and rapid training without compromising visual fidelity. However, 3DGS relies heavily on accurate camera poses and high-quality point cloud initialization, which are difficult to obtain in sparse-view scenarios. While traditional Structure from Motion (SfM) pipelines often fail in these settings, existing learning-based point estimation alternatives typically require reliable reference views and remain sensitive to pose or depth errors. In this work, we propose a robust method utilizing π^3, a reference-free point cloud estimation network. We integrate dense initialization from π^3 with a regularization scheme designed to mitigate geometric inaccuracies. Specifically, we employ uncertainty-guided depth supervision, normal consistency loss, and depth warping. Experimental results demonstrate that our approach achieves state-of-the-art performance on the Tanks and Temples, LLFF, DTU, and MipNeRF360 datasets.
MeshSplats: Mesh-Based Rendering with Gaussian Splatting Initialization
Faculty of Mathematics and Computer Science, Jagiellonian University in Kraków(克拉科夫雅盖隆大学数学与计算机科学学院);Department of Engineering, University of Cambridge(剑桥大学工程系);IDEAS Research Institute(IDEAS研究所)
3D Gaussian Splatting with Fisheye Images: Field of View Analysis and Depth-Based Initialization
Department of Computing Sciences, Tampere University(塔尔基大学计算机科学系);Department of Computer Science, Aalto University(阿尔托大学计算机科学系);Department of Computer Science and Engineering, University of Oulu(奥卢大学计算机科学与工程系)
The Role of Initialization in 3D Gaussian Splatting
Czech Technical University in Prague, Faculty of Electrical Engineering(布拉格捷克技术大学电子工程系);Czech Technical University in Prague, Czech Institute of Informatics, Robotics and Cybernetics(布拉格捷克技术大学信息技术、机器人学与自动化研究所)
Segmentation-Driven Initialization for Sparse-view 3D Gaussian Splatting
Mid Sweden University(中瑞典大学);Technical University of Berlin(柏林技术大学);Hochschule für Technik und Wirtschaft Berlin(柏林技术与经济高等学院);Department of Computer and Electrical Engineering, Mid Sweden University(中瑞典大学计算机与电气工程系);Department of Telecommunication Systems, Technical University of Berlin(柏林技术大学电信系统系)
Physically Plausible Human-Object Rendering from Sparse Views via 3D Gaussian Splatting
College of Computer Science, Zhejiang University(浙江大学计算机科学与技术学院);Department of Computer Science and Engineering, The Hong Kong University of Science and Technology(香港科技大学计算机科学与工程系)
SolidGS: Consolidating Gaussian Surfel Splatting for Sparse-View Surface Reconstruction
Texas A&M University(德克萨斯农工大学);OPPO US Research Center(OPPO美国研究中心);Nanyang Technological University(南洋理工大学);Hong Kong University of Science and Technology(香港科技大学);The University of Hong Kong(香港大学)
GeoSplat: A Deep Dive into Geometry-Constrained Gaussian Splatting
Department of Applied Mathematics and Theoretical Physics, University of Cambridge(应用数学与理论物理系,剑桥大学);Institut Camille Jordan, Université Claude Bernard Lyon 1(让·Camille Jordan研究所,克莱蒙费朗斯大学)
SparseStreet: Sparse Gaussian Splatting for Real-Time Street Scene Simulation
Peking University(北京大学);Chinese Academy of Sciences(中国科学院);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Autonomous Driving Development, NIO(蔚来自动驾驶开发)
AD-GS: Anatomical Density-guided 3D Gaussian Splatting for Sparse-View Dental CBCT Reconstruction
Beijing University of Posts and Telecommunications(北京邮电大学);Peking University Third Hospital(北京大学第三医院);National Institute of Informatics(信息学研究所);Peking University(北京大学)
Learning Fine-Grained Geometry for Sparse-View Splatting via Cascade Depth Loss
School of Computer Science The University of Sydney Sydney, Australia;School of Computer Science University of Bristol Bristol, United Kingdom;School of Science Edith Cowan University Perth, Australia
LGDWT-GS: Local and Global Discrete Wavelet-Regularized 3D Gaussian Splatting for Sparse-View Scene Reconstruction
National Institute of Standards and Technology(国家标准与技术研究院);Department of Physics, Colorado State University(科罗拉多州立大学物理系);Electrical Engineering Department, University of Colorado(科罗拉多大学电气工程系)
SparseLGS: Sparse View Language Embedded Gaussian Splatting
School of Mathematical Science, University of Science and Technology of China(科学技术大学数学科学学院);Meta;Apple Inc.(苹果公司);Alpha Labs at Goertek(Goertek的Alpha实验室)