3D Gaussian Splatting在实时神经渲染中获得了广泛关注与应用。同时,人们也对该技术在点云存储、性能以及稀疏视角下的鲁棒性等方面的局限性提出了担忧,从而催生了各种改进方法。然而,对于splatting本身固有的局部仿射近似所引入的投影误差这一基本问题,以及这些误差对照片级真实感渲染质量的影响,一直明显缺乏关注。本文探讨了3D Gaussian Splatting的投影误差函数,从投影函数一阶泰勒展开的残差误差入手。分析建立了误差与高斯均值位置之间的相关性。随后,利用函数优化理论,本文分析了该函数的极小值,为Gaussian Splatting提供了一种最优投影策略,称为Optimal Gaussian Splatting,该策略能适应多种相机模型。实验验证进一步证实,这种投影方法减少了伪影,从而产生更具说服力的真实感渲染。
英文摘要:
3D Gaussian Splatting has garnered extensive attention and application in real-time neural rendering. Concurrently, concerns have been raised about the limitations of this technology in aspects such as point cloud storage, performance, and robustness in sparse viewpoints, leading to various improvements. However, there has been a notable lack of attention to the fundamental problem of projection errors introduced by the local affine approximation inherent in the splatting itself, and the consequential impact of these errors on the quality of photo-realistic rendering. This paper addresses the projection error function of 3D Gaussian Splatting, commencing with the residual error from the first-order Taylor expansion of the projection function. The analysis establishes a correlation between the error and the Gaussian mean position. Subsequently, leveraging function optimization theory, this paper analyzes the function's minima to provide an optimal projection strategy for Gaussian Splatting referred to Optimal Gaussian Splatting, which can accommodate a variety of camera models. Experimental validation further confirms that this projection methodology reduces artifacts, resulting in a more convincingly realistic rendering.
Fisheye-GS: Lightweight and Extensible Gaussian Splatting Module for Fisheye Cameras
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);The Chinese University of Hong Kong(香港中文大学);Hong Kong University of Science and Technology(香港科技大学)
Computer vision training dataset generation for robotic environments using Gaussian splatting
Institute of Engineering and Technology, Faculty of Physics, Astronomy and Informatics, Nicolaus Copernicus University in Toruń(工程与技术学院,物理、天文学与信息学学院,尼古拉·哥白尼大学)
AD-GS: Anatomical Density-guided 3D Gaussian Splatting for Sparse-View Dental CBCT Reconstruction
Beijing University of Posts and Telecommunications(北京邮电大学);Peking University Third Hospital(北京大学第三医院);National Institute of Informatics(信息学研究所);Peking University(北京大学)
You Need a Transition Plane: Bridging Continuous Panoramic 3D Reconstruction with Perspective Gaussian Splatting
Institute of Information Science, Beijing Jiaotong University(北京交通大学信息科学研究所);Visual Intelligence +X International Cooperation Joint Laboratory of MOE(教育部视觉智能+X国际合作联合实验室);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院)
A Mixed-Primitive-based Gaussian Splatting Method for Surface Reconstruction
Lancaster University, United Kingdom(兰卡斯特大学,英国);The University of Queensland, Australia(昆士兰大学,澳大利亚);The Hong Kong Polytechnic University, China(香港理工大学,中国);University at Buffalo, United States of America(布法罗大学,美国)
Underwater360: Reconstructing Underwater Scenes from Panoramic Images with Omnidirectional Gaussian Splatting
School of Software, Dalian University of Technology(大连理工大学软件学院);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院)
Hunan University(湖南大学);Nanyang Technological University(南洋理工大学);Lushan Innovation Lab(庐山创新实验室);Key Laboratory of Digital Culture Smart Design Technology, Minstry of Culture and Tourism(文化与旅游部数字文化智能设计技术重点实验室)
GSM-GS: Geometry-Constrained Single and Multi-view Gaussian Splatting for Surface Reconstruction
School of Automation and Intelligent Manufacturing, Southern University Science and Technology(自动化与智能制造学院,南方科技大学);Research Institute of Mine Artificial Intelligence, China Coal Research Institute(煤矿人工智能研究院,中国煤炭研究总院);Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University(人工智能与机器人研究院,西安交通大学)
University of Liverpool(利物浦大学);Xi’an Jiaotong-Liverpool University(西交利物浦大学);Universidad Antonio de Nebrija(安东尼奥·德·内布里哈大学);ARIES Research Centre(ARIES研究中心)