细节对室内开放词汇3D实例分割至关重要
Details Matter for Indoor Open-vocabulary 3D Instance Segmentation
AI总结：
针对开放词汇3D实例分割任务，本研究整合互补思路，提出包含基于3D跟踪的候选框聚合、Alpha-CLIP分类及SMS分数归一化的两阶段框架，在多个基准上实现了超越端到端封闭词汇方法的最优性能。
摘要：
与通常采用端到端训练的封闭词汇3D实例分割不同，开放词汇3D实例分割（OV-3DIS）通常利用视觉语言模型（VLM）生成3D实例候选框并对其进行分类。尽管现有研究已提出多种不同思路，但我们发现这些独立思路并非互斥，而是具有互补性。本文中，我们通过精心设计一套方案将这些思路结合起来并加以优化以应对核心挑战，提出了一种全新的OV-3DIS最优解决方案。我们的方案遵循两阶段框架：3D候选框生成和实例分类。我们采用基于鲁棒3D跟踪的候选框聚合方法生成3D候选框，并通过迭代合并/移除操作去除重叠或不完整的候选框。在分类阶段，我们用Alpha-CLIP替换标准CLIP模型，该模型将对象掩码作为alpha通道纳入，以减少背景噪声并获得以对象为中心的表征。此外，我们引入了标准化最大相似度（SMS）分数来归一化文本与候选框的相似度，有效过滤误报并提升精确率。我们的框架在ScanNet200和S3DIS数据集的所有AP和AR指标上均达到了最优性能，甚至超过了一种端到端封闭词汇方法。
英文摘要：
Unlike closed-vocabulary 3D instance segmentation that is often trained end-to-end, open-vocabulary 3D instance segmentation (OV-3DIS) often leverages vision-language models (VLMs) to generate 3D instance proposals and classify them. While various concepts have been proposed from existing research, we observe that these individual concepts are not mutually exclusive but complementary. In this paper, we propose a new state-of-the-art solution for OV-3DIS by carefully designing a recipe to combine the concepts together and refining them to address key challenges. Our solution follows the two-stage scheme: 3D proposal generation and instance classification. We employ robust 3D tracking-based proposal aggregation to generate 3D proposals and remove overlapped or partial proposals by iterative merging/removal. For the classification stage, we replace the standard CLIP model with Alpha-CLIP, which incorporates object masks as an alpha channel to reduce background noise and obtain object-centric representation. Additionally, we introduce the standardized maximum similarity (SMS) score to normalize text-to-proposal similarity, effectively filtering out false positives and boosting precision. Our framework achieves state-of-the-art performance on ScanNet200 and S3DIS across all AP and AR metrics, even surpassing an end-to-end closed-vocabulary method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
