arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2412.05908cs.CV

GBR:用于高保真Gaussian Splatting与增强Mesh重建的生成式Bundle Refinement

GBR: Generative Bundle Refinement for High-fidelity Gaussian Splatting with Enhanced Mesh Reconstruction

College of future information technology, Fudan University(未来信息科技学院,复旦大学);School of Biomedical Engineering, Tsinghua University(生物医学工程学院,清华大学);Key Laboratory for Information Science of Electromagnetic Waves (MoE), Fudan University(电磁波信息科学重点实验室(MoE),复旦大学);Department of Automation, Tsinghua University(自动化系,清华大学);MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University(人工智能重点实验室,人工智能研究院,上海交通大学)

Jianing Zhang, Yuchao Zheng, Ziwei Li, Qionghai Dai, Xiaoyun Yuan

AI总结:

提出GBR方法,通过neural bundle adjustment和生成式深度精修模块,结合多模态损失函数,在仅4-6个稀疏视图下实现高保真Gaussian splatting和网格重建,显著优于现有方法。

摘要:

Gaussian splatting因使用连续Gaussian基元对3D场景进行高效表示和渲染而受到关注。然而,由于几何和光度信息有限,它在稀疏视图输入下难以处理,导致深度、形状和纹理上的歧义。我们提出了GBR: Generative Bundle Refinement,一种仅使用4-6个输入视图进行高保真Gaussian splatting和网格化的方法。GBR集成了一个neural bundle adjustment模块以增强几何准确性,以及一个生成式深度精修模块以提升几何保真度。具体而言,neural bundle adjustment模块集成基础网络从未定位图像中生成初始3D点图和点匹配,随后进行bundle adjustment优化,以改善多视图一致性和点云精度。生成式深度精修模块采用基于扩散的策略,在保持尺度的同时增强几何细节和保真度。最后,针对Gaussian splatting优化,我们提出了一种多模态损失函数,结合深度和法线一致性、几何正则化以及伪视图监督,在稀疏视图条件下提供稳健指导。在广泛使用的数据集上的实验表明,GBR在稀疏视图输入下显著优于现有方法。此外,GBR展示了重建和渲染大规模真实世界场景的能力,如滕王阁和长城,仅使用6个视图即可获得显著细节。

英文摘要:

Gaussian splatting has gained attention for its efficient representation and rendering of 3D scenes using continuous Gaussian primitives. However, it struggles with sparse-view inputs due to limited geometric and photometric information, causing ambiguities in depth, shape, and texture. we propose GBR: Generative Bundle Refinement, a method for high-fidelity Gaussian splatting and meshing using only 4-6 input views. GBR integrates a neural bundle adjustment module to enhance geometry accuracy and a generative depth refinement module to improve geometry fidelity. More specifically, the neural bundle adjustment module integrates a foundation network to produce initial 3D point maps and point matches from unposed images, followed by bundle adjustment optimization to improve multiview consistency and point cloud accuracy. The generative depth refinement module employs a diffusion-based strategy to enhance geometric details and fidelity while preserving the scale. Finally, for Gaussian splatting optimization, we propose a multimodal loss function incorporating depth and normal consistency, geometric regularization, and pseudo-view supervision, providing robust guidance under sparse-view conditions. Experiments on widely used datasets show that GBR significantly outperforms existing methods under sparse-view inputs. Additionally, GBR demonstrates the ability to reconstruct and render large-scale real-world scenes, such as the Pavilion of Prince Teng and the Great Wall, with remarkable details using only 6 views.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑