SceneSplat：基于高斯泼溅与视觉-语言预训练的场景理解
SceneSplat: Gaussian Splatting-based Scene Understanding with Vision-Language Pretraining
AI总结：
本文提出SceneSplat，首个原生基于3DGS的大规模室内场景理解方法，并构建SceneSplat-7K数据集，通过自监督学习实现3D语义推理。
摘要：
识别任意或未见过的类别对于全面的真实世界3D场景理解至关重要。目前，所有现有方法在训练阶段或推理阶段都依赖2D或文本模态。这凸显了明显缺乏一种能够仅处理3D数据以端到端学习语义的模型，以及训练此类模型所需的数据。与此同时，3D高斯泼溅（3DGS）已成为各种视觉任务中3D场景表示的事实标准。然而，以可泛化的方式将语义推理有效集成到3DGS中仍然是一个开放性挑战。为了解决这些局限性，我们提出了SceneSplat，据我们所知，这是第一个原生运行在3DGS上的大规模3D室内场景理解方法。此外，我们提出了一种自监督学习方案，能够从无标注场景中解锁丰富的3D特征学习。为了支撑所提出的方法，我们引入了SceneSplat-7K，这是第一个面向室内场景的大规模3DGS数据集，包含7916个场景，来源于七个既有数据集，如ScanNet和Matterport3D。生成SceneSplat-7K所需的计算资源相当于在L4 GPU上运行150个GPU日，从而为基于3DGS的室内场景推理提供了标准化基准。我们在SceneSplat-7K上的详尽实验证明了所提方法相较于既有基线具有显著优势。
英文摘要：
Recognizing arbitrary or previously unseen categories is essential for comprehensive real-world 3D scene understanding. Currently, all existing methods rely on 2D or textual modalities during training or together at inference. This highlights the clear absence of a model capable of processing 3D data alone for learning semantics end-to-end, along with the necessary data to train such a model. Meanwhile, 3D Gaussian Splatting (3DGS) has emerged as the de facto standard for 3D scene representation across various vision tasks. However, effectively integrating semantic reasoning into 3DGS in a generalizable manner remains an open challenge. To address these limitations, we introduce SceneSplat, to our knowledge the first large-scale 3D indoor scene understanding approach that operates natively on 3DGS. Furthermore, we propose a self-supervised learning scheme that unlocks rich 3D feature learning from unlabeled scenes. To power the proposed methods, we introduce SceneSplat-7K, the first large-scale 3DGS dataset for indoor scenes, comprising 7916 scenes derived from seven established datasets, such as ScanNet and Matterport3D. Generating SceneSplat-7K required computational resources equivalent to 150 GPU days on an L4 GPU, enabling standardized benchmarking for 3DGS-based reasoning for indoor scenes. Our exhaustive experiments on SceneSplat-7K demonstrate the significant benefit of the proposed method over the established baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
