SegSplat: 用于快速前馈3D重建与丰富开放词汇语义理解的框架
SegSplat: Feed-forward Gaussian Splatting and Open-Set Semantic Segmentation
- ETH Zürich（苏黎世联邦理工学院）
- Google（谷歌）
- Microsoft（微软）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
SegSplat通过单次传递实现高效3D重建与开放集语义分割，无需每场景优化。
AI中文摘要：
我们引入了SegSplat，一种新的框架，旨在弥合快速前馈3D重建与丰富、开放词汇语义理解之间的差距。通过从多视角2D基础模型特征中构建紧凑的语义记忆库，并在单次传递中为每个3D高斯预测离散语义索引以及几何和外观属性，SegSplat高效地为场景注入可查询的语义。我们的实验表明，SegSplat在几何保真度上与最先进的前馈3D高斯喷射方法相当，同时能够实现稳健的开放集语义分割，关键在于不需要任何针对语义特征整合的每场景优化。这项工作代表了迈向实用、即时生成语义感知3D环境的重要一步，对于推进机器人交互、增强现实和其他智能系统具有关键作用。
英文摘要：
We have introduced SegSplat, a novel framework designed to bridge the gap between rapid, feed-forward 3D reconstruction and rich, open-vocabulary semantic understanding. By constructing a compact semantic memory bank from multi-view 2D foundation model features and predicting discrete semantic indices alongside geometric and appearance attributes for each 3D Gaussian in a single pass, SegSplat efficiently imbues scenes with queryable semantics. Our experiments demonstrate that SegSplat achieves geometric fidelity comparable to state-of-the-art feed-forward 3D Gaussian Splatting methods while simultaneously enabling robust open-set semantic segmentation, crucially \textit{without} requiring any per-scene optimization for semantic feature integration. This work represents a significant step towards practical, on-the-fly generation of semantically aware 3D environments, vital for advancing robotic interaction, augmented reality, and other intelligent systems.
