GSemSplat：从未标定图像对中可泛化的语义3D高斯泼溅
GSemSplat: Generalizable Semantic 3D Gaussian Splatting from Uncalibrated Image Pairs
AI总结：
提出GSemSplat框架，从未标定稀疏图像对学习开放词汇3D语义表示，无需逐场景优化，在ScanNet++上优于传统方法。
摘要：
对3D世界的建模与理解对从增强现实到机器人导航的各种应用至关重要。基于3D Gaussian Splatting的最新进展已将多视图图像中的语义信息集成到高斯基元中。然而，这些方法通常需要从密集标定图像进行昂贵的逐场景优化，限制了其实用性。在本文中，我们考虑从稀疏、未标定图像对进行可泛化3D语义场建模的新任务。基于Splatt3R架构，我们引入GSemSplat，这是一个无需逐场景优化、密集图像收集或标定，即可学习与3D高斯关联的开放词汇语义表示的框架。为确保在3D空间中有效可靠地学习语义特征，我们采用双特征方法，利用区域特定和上下文感知语义特征作为2D空间的监督。这使我们能利用它们互补的优势。在ScanNet++数据集上的实验结果证明了我们的方法相比传统特定场景方法的有效性和优越性。我们希望我们的工作能激发更多关于可泛化3D理解的研究。
英文摘要：
Modeling and understanding the 3D world is crucial for various applications, from augmented reality to robotic navigation. Recent advancements based on 3D Gaussian Splatting have integrated semantic information from multi-view images into Gaussian primitives. However, these methods typically require costly per-scene optimization from dense calibrated images, limiting their practicality. In this paper, we consider the new task of generalizable 3D semantic field modeling from sparse, uncalibrated image pairs. Building upon the Splatt3R architecture, we introduce GSemSplat, a framework that learns open-vocabulary semantic representations linked to 3D Gaussians without the need for per-scene optimization, dense image collections or calibration. To ensure effective and reliable learning of semantic features in 3D space, we employ a dual-feature approach that leverages both region-specific and context-aware semantic features as supervision in the 2D space. This allows us to capitalize on their complementary strengths. Experimental results on the ScanNet++ dataset demonstrate the effectiveness and superiority of our approach compared to the traditional scene-specific method. We hope our work will inspire more research into generalizable 3D understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
