arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2403.07494cs.ROcs.CV

SemGauss-SLAM:稠密语义高斯溅射SLAM

SemGauss-SLAM: Dense Semantic Gaussian Splatting SLAM

School of Automation and Intelligent Sensing, Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai Jiao Tong University, Shanghai(自动化与智能感知学院、系统控制与信息处理重点实验室、中华人民共和国教育部、上海交通大学、上海);University of Cambridge, United Kingdom(剑桥大学、英国)

Siting Zhu, Renjie Qin, Guangming Wang, Jiuming Liu, Hesheng Wang

AI总结:

提出SemGauss-SLAM,将语义特征嵌入3D高斯表示,结合特征级损失与语义束调整,实现高精度稠密语义建图、稳健跟踪与高质量渲染,在Replica和ScanNet上优于现有辐射场SLAM方法。

摘要:

我们提出了SemGauss-SLAM,一个利用3D高斯表示的稠密语义SLAM系统,能够同时实现精确的3D语义建图、稳健的相机跟踪和高品质渲染。在该系统中,我们将语义特征嵌入到3D高斯表示中,有效地在环境的空间布局内编码语义信息,以实现精确的语义场景表示。此外,我们提出了特征级损失用于更新3D高斯表示,从而为3D高斯优化提供更高级别的指导。另外,为了减少跟踪中的累积漂移并提高语义重建精度,我们引入了语义信息感知的束调整。通过利用多帧语义关联,该策略实现了3D高斯表示与相机位姿的联合优化,从而获得低漂移跟踪和精确的语义建图。我们的SemGauss-SLAM在Replica和ScanNet数据集上的建图和跟踪精度方面优于现有的基于辐射场的SLAM方法,同时在高精度语义分割和稠密语义建图方面也展现出卓越的能力。

英文摘要:

We propose SemGauss-SLAM, a dense semantic SLAM system utilizing 3D Gaussian representation, that enables accurate 3D semantic mapping, robust camera tracking, and high-quality rendering simultaneously. In this system, we incorporate semantic feature embedding into 3D Gaussian representation, which effectively encodes semantic information within the spatial layout of the environment for precise semantic scene representation. Furthermore, we propose feature-level loss for updating 3D Gaussian representation, enabling higher-level guidance for 3D Gaussian optimization. In addition, to reduce cumulative drift in tracking and improve semantic reconstruction accuracy, we introduce semantic-informed bundle adjustment. By leveraging multi-frame semantic associations, this strategy enables joint optimization of 3D Gaussian representation and camera poses, resulting in low-drift tracking and accurate semantic mapping. Our SemGauss-SLAM demonstrates superior performance over existing radiance field-based SLAM methods in terms of mapping and tracking accuracy on Replica and ScanNet datasets, while also showing excellent capabilities in high-precision semantic segmentation and dense semantic mapping.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑