arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2503.06235cs.CV

StreamGS:面向未位姿图像流的在线可泛化高斯泼溅重建

StreamGS: Online Generalizable Gaussian Splatting Reconstruction for Unposed Image Streams

Microsoft Research Asia(微软亚洲研究院);HKUST(香港科技大学);HKUST(GZ)(香港科技大学(广州))

Yang LI, Jinglu Wang, Lei Chu, Xiao Li, Shiu-hong Kao, Ying-Cong Chen, Yan Lu

AI总结:

针对未位姿图像流在线3DGS重建的三大挑战,提出StreamGS方法,通过逐帧预测聚合高斯、内容自适应细化及跨帧冗余合并,实现质量相当、速度快150倍且泛化性更优的在线重建。

摘要:

3D高斯泼溅(3D Gaussian Splatting, 3DGS)的出现推动了3D场景重建与新视角合成的发展。随着需要即时反馈的交互式应用日益受到关注,实时在线3DGS重建需求迫切。但现有方法均无法满足该需求,主要面临三大挑战:缺乏预定义相机参数、需要可泛化的3DGS优化、必须降低冗余。我们提出StreamGS,一种面向未位姿图像流的在线可泛化3DGS重建方法,通过预测并聚合逐帧高斯,逐步将图像流转换为3D高斯流。本方法引入内容自适应细化,克服了初始点重建方法Dust3r在处理域外(OOD)问题时的局限性。该细化过程通过在相邻帧间建立可靠的像素对应关系增强跨帧一致性,此类对应关系进一步通过跨帧特征聚合辅助合并冗余高斯,从而降低高斯密度,大幅降低计算与内存成本,支撑在线重建。在多类数据集上的大量实验表明,StreamGS的重建质量与基于优化的方法相当,但速度快150倍,且在处理OOD场景时表现出更优的泛化能力。

英文摘要:

The advent of 3D Gaussian Splatting (3DGS) has advanced 3D scene reconstruction and novel view synthesis. With the growing interest of interactive applications that need immediate feedback, online 3DGS reconstruction in real-time is in high demand. However, none of existing methods yet meet the demand due to three main challenges: the absence of predetermined camera parameters, the need for generalizable 3DGS optimization, and the necessity of reducing redundancy. We propose StreamGS, an online generalizable 3DGS reconstruction method for unposed image streams, which progressively transform image streams to 3D Gaussian streams by predicting and aggregating per-frame Gaussians. Our method overcomes the limitation of the initial point reconstruction \cite{dust3r} in tackling out-of-domain (OOD) issues by introducing a content adaptive refinement. The refinement enhances cross-frame consistency by establishing reliable pixel correspondences between adjacent frames. Such correspondences further aid in merging redundant Gaussians through cross-frame feature aggregation. The density of Gaussians is thereby reduced, empowering online reconstruction by significantly lowering computational and memory costs. Extensive experiments on diverse datasets have demonstrated that StreamGS achieves quality on par with optimization-based approaches but does so 150 times faster, and exhibits superior generalizability in handling OOD scenes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑