开放词汇与指代分割：利用2D检测器实现3D高斯
Open-Vocabulary and Referring Segmentation for 3D Gaussians Using 2D Detectors
AI总结：
提出GaussDet方法，利用离散的2D开放词汇检测器与指代表达能力，通过视图聚合语义标签分布实现3D场景的开放词汇分割和指代分割，在零样本设置下显著提升指代分割性能。
摘要：
3D高斯泼溅（3DGS）已成为3D场景重建的前沿技术。将3DGS扩展为语言驱动的开放词汇理解，在具身AI等实际应用中受到广泛关注。现有方法通过学习实例特征属性，并将高维对比语言-图像预训练（CLIP）特征直接蒸馏到场景表示中，从而分配语义。然而，这些方法的实例分组机制要么需要预定义的实例数量，要么在自底向上的分组策略中受到噪声影响。此外，对CLIP的依赖将语义理解限制在简单的名词短语上，阻碍了复杂的空间推理和指代表达基础。我们提出GaussDet，该方法通过利用具有指代表达能力的离散开放词汇2D物体检测器，避免了密集CLIP特征的需求。我们为单个高斯学习实例特征，将场景分解为3D实例组。通过渲染这些组并从多视图2D检测中聚合语义投票，我们为每个3D实例生成鲁棒的视图聚合语义标签分布（VASD）。这种视图聚合策略充当强正则化器，减弱由低质量实例分组引起的虚假标签。我们的方法实现了从简单语言查询到复杂指代基础的直接零样本扩展。在两个关键任务——开放词汇分割（LeRF-OVS, ScanNet）和指代表达基础（Ref-LeRF）上的广泛评估表明，GaussDet在现有方法上取得了一致的改进。最值得注意的是，在严格的零样本设置下，我们在指代基础方面实现了16.7% mIoU的显著提升。
英文摘要：
3D Gaussian Splatting (3DGS) has emerged at the forefront of 3D scene reconstruction. Extending 3DGS with language-driven, open-vocabulary understanding has gained significant attention for real-world applications such as embodied AI. Recent methods achieve this by learning an instance feature attribute and assigning semantics by distilling high-dimensional Contrastive Language-Image Pretraining (CLIP) features directly into the scene representation. However, the instance grouping mechanisms of these methods either require a predefined number of instances or suffer from noise in their bottom-up grouping strategies. Furthermore, the reliance on CLIP restricts semantic understanding to simple noun phrases, preventing complex spatial reasoning and referential expression grounding. We present GaussDet, a method that circumvents the need for dense CLIP features by leveraging discrete, open-vocabulary 2D object detectors with referring expression capabilities. We learn instance features for individual Gaussians to decompose the scene into 3D instance groups. By rendering these groups and aggregating semantic votes from multi-view 2D detections, we generate a robust View-Aggregated Semantic Label Distribution (VASD) for each 3D instance. This view-aggregation strategy acts as a strong regularizer, attenuating spurious labels caused by low-quality instance grouping. Our approach enables a straightforward, zero-shot extension from simple language queries to complex referential grounding. Extensive evaluations across two key tasks -- open-vocabulary segmentation (LeRF-OVS, ScanNet) and referring expression grounding (Ref-LeRF) -- demonstrate that GaussDet achieves consistent improvements over existing methods. Most notably, we achieve a substantial 16.7% mIoU improvement in referential grounding within a strict zero-shot setting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
