arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.08331cs.CV

SLGaussian:稀疏视图下的快速语言高斯泼溅

SLGaussian: Fast Language Gaussian Splatting in Sparse Views

Tsinghua Shenzhen International Graduate School(清华大学深圳国际研究生院)

Kangjie Chen, BingQuan Dai, Minghan Qin, Dongbin Zhang, Peihao Li, Yingshuang Zou, Haoqian Wang

AI总结:

SLGaussian提出一种前馈方法,从稀疏视图直接构建3D语义场,通过视频跟踪保持SAM分割一致性并用低维索引嵌入CLIP特征,在LERF和3D-OVS数据集上以更快速度超越现有方法。

摘要:

3D语义场学习对于自主导航、AR/VR和机器人等应用至关重要,在这些应用中,从有限视点准确理解3D场景是必不可少的。现有方法在稀疏视图条件下表现不佳,依赖于低效的逐场景多视图优化,这对许多现实世界任务而言并不实用。为了解决这一问题,我们提出了SLGaussian,一种从稀疏视点构建3D语义场的前馈方法,能够直接推理基于3DGS的场景。通过视频跟踪确保SAM分割的一致性,并使用低维索引表示高维CLIP特征,SLGaussian高效地将语言信息嵌入3D空间,为稀疏视图条件下的精确3D场景理解提供了稳健的解决方案。在LERF和3D-OVS数据集上的双视图稀疏3D对象查询与分割实验中,SLGaussian在选定的IoU、定位精度和mIoU指标上均优于现有方法。此外,我们的模型在30秒内完成场景推理,每次开放词汇查询仅需0.011秒。

英文摘要:

3D semantic field learning is crucial for applications like autonomous navigation, AR/VR, and robotics, where accurate comprehension of 3D scenes from limited viewpoints is essential. Existing methods struggle under sparse view conditions, relying on inefficient per-scene multi-view optimizations, which are impractical for many real-world tasks. To address this, we propose SLGaussian, a feed-forward method for constructing 3D semantic fields from sparse viewpoints, allowing direct inference of 3DGS-based scenes. By ensuring consistent SAM segmentations through video tracking and using low-dimensional indexing for high-dimensional CLIP features, SLGaussian efficiently embeds language information in 3D space, offering a robust solution for accurate 3D scene understanding under sparse view conditions. In experiments on two-view sparse 3D object querying and segmentation in the LERF and 3D-OVS datasets, SLGaussian outperforms existing methods in chosen IoU, Localization Accuracy, and mIoU. Moreover, our model achieves scene inference in under 30 seconds and open-vocabulary querying in just 0.011 seconds per query.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑