OpenGS-Fusion：基于混合三维高斯泼溅的开放词汇密集建图，用于精细化对象级理解
OpenGS-Fusion: Open-Vocabulary Dense Mapping with Hybrid 3D Gaussian Splatting for Refined Object-Level Understanding
AI总结：
OpenGS-Fusion提出融合三维高斯与截断符号距离场的开放词汇密集建图框架，结合MLLM辅助自适应阈值化，提升三维对象分割精度17% mIoU，优于现有方法。
摘要：
近年来，三维场景理解方面的进展在实现使用开放词汇查询与场景交互方面取得了显著进步，特别是在虚拟现实/增强现实和机器人应用中。然而，现有方法受限于僵化的离线流程，并且无法针对开放式查询提供精确的三维对象级理解。在本文中，我们提出了OpenGS-Fusion，一种创新的开放词汇密集建图框架，它改进了语义建模并细化了对象级理解。OpenGS-Fusion将三维高斯表示与截断符号距离场相结合，以促进语义特征的即时无损融合。此外，我们引入了一种新颖的多模态语言引导方法，称为MLLM辅助自适应阈值化，通过自适应调整相似度阈值来细化三维对象的分割，与固定阈值策略相比，三维mIoU提升了17%。大量实验表明，我们的方法在三维对象理解和场景重建质量方面优于现有方法，并展示了其在语言引导的场景交互中的有效性。代码可在https://young-bit.github.io/opengs-fusion.github.io/获取。
英文摘要：
Recent advancements in 3D scene understanding have made significant strides in enabling interaction with scenes using open-vocabulary queries, particularly for VR/AR and robotic applications. Nevertheless, existing methods are hindered by rigid offline pipelines and the inability to provide precise 3D object-level understanding given open-ended queries. In this paper, we present OpenGS-Fusion, an innovative open-vocabulary dense mapping framework that improves semantic modeling and refines object-level understanding. OpenGS-Fusion combines 3D Gaussian representation with a Truncated Signed Distance Field to facilitate lossless fusion of semantic features on-the-fly. Furthermore, we introduce a novel multimodal language-guided approach named MLLM-Assisted Adaptive Thresholding, which refines the segmentation of 3D objects by adaptively adjusting similarity thresholds, achieving an improvement 17\% in 3D mIoU compared to the fixed threshold strategy. Extensive experiments demonstrate that our method outperforms existing methods in 3D object understanding and scene reconstruction quality, as well as showcasing its effectiveness in language-guided scene interaction. The code is available at https://young-bit.github.io/opengs-fusion.github.io/ .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
