SemGS：从稀疏视图中进行前馈语义3D高斯点云分割以实现可泛化的场景理解
SemGS: Feed-Forward Semantic 3D Gaussian Splatting from Sparse Views for Generalizable Scene Understanding
AI总结：
SemGS通过双分支架构和相机感知注意力机制，从稀疏视图中高效重建可泛化的语义3D场，实现快速推理和广泛适用性。
摘要：
语义理解3D场景对于机器人在复杂环境中有效和安全地操作至关重要。现有的语义场景重建和语义感知新视角合成方法往往依赖于密集多视角输入，并需要场景特定的优化，限制了它们在现实应用中的实用性和可扩展性。为了解决这些挑战，我们提出了SemGS，一种从稀疏图像输入中重建可泛化的语义场的前馈框架。SemGS使用双分支架构提取颜色和语义特征，其中两个分支共享浅层CNN层，使语义推理能够利用颜色外观中的纹理和结构线索。我们还将在特征提取器中引入了相机感知的注意力机制，以显式建模相机视角之间的几何关系。提取的特征被解码为双高斯，这些高斯在保持分支特定属性的同时共享几何一致性，并进一步细化以在新视角下合成语义地图。此外，我们引入了区域平滑损失以增强语义一致性。实验表明，SemGS在基准数据集上实现了最先进的性能，同时在多样化的合成和现实场景中提供了快速推理和强大的泛化能力。
英文摘要：
Semantic understanding of 3D scenes is essential for robots to operate effectively and safely in complex environments. Existing methods for semantic scene reconstruction and semantic-aware novel view synthesis often rely on dense multi-view inputs and require scene-specific optimization, limiting their practicality and scalability in real-world applications. To address these challenges, we propose SemGS, a feed-forward framework for reconstructing generalizable semantic fields from sparse image inputs. SemGS uses a dual-branch architecture to extract color and semantic features, where the two branches share shallow CNN layers, allowing semantic reasoning to leverage textural and structural cues in color appearance. We also incorporate a camera-aware attention mechanism into the feature extractor to explicitly model geometric relationships between camera viewpoints. The extracted features are decoded into dual-Gaussians that share geometric consistency while preserving branch-specific attributes, and further rasterized to synthesize semantic maps under novel viewpoints. Additionally, we introduce a regional smoothness loss to enhance semantic coherence. Experiments show that SemGS achieves state-of-the-art performance on benchmark datasets, while providing rapid inference and strong generalization capabilities across diverse synthetic and real-world scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
