OpenMask3D：开放词汇3D实例分割
OpenMask3D: Open-Vocabulary 3D Instance Segmentation
AI总结：
本文提出OpenMask3D，一种零样本开放词汇3D实例分割方法，通过类别无关掩码引导和多视图CLIP特征融合，在ScanNet200和Replica上优于现有方法，尤其在长尾分布上表现突出。
摘要：
我们引入了开放词汇3D实例分割这一任务。当前的3D实例分割方法通常只能识别训练数据集中标注的预定义封闭类别集合中的物体类别。这给现实世界应用带来了重要限制，因为在这些应用中，人们可能需要根据与各种物体相关的新颖开放词汇查询来执行任务。近年来，开放词汇3D场景理解方法通过为场景中的每个点学习可查询特征来应对这一问题。虽然这种表示可以直接用于执行语义分割，但现有方法无法分离多个物体实例。在本工作中，我们针对这一局限性提出了OpenMask3D，这是一种用于开放词汇3D实例分割的零样本方法。在预测的类别无关3D实例掩码的引导下，我们的模型通过基于CLIP的图像嵌入的多视图融合来聚合每个掩码的特征。在ScanNet200和Replica上的实验和消融研究表明，OpenMask3D优于其他开放词汇方法，尤其是在长尾分布上。定性实验进一步展示了OpenMask3D能够根据描述几何形状、可供性和材料的自由形式查询来分割物体属性的能力。
英文摘要：
We introduce the task of open-vocabulary 3D instance segmentation. Current approaches for 3D instance segmentation can typically only recognize object categories from a pre-defined closed set of classes that are annotated in the training datasets. This results in important limitations for real-world applications where one might need to perform tasks guided by novel, open-vocabulary queries related to a wide variety of objects. Recently, open-vocabulary 3D scene understanding methods have emerged to address this problem by learning queryable features for each point in the scene. While such a representation can be directly employed to perform semantic segmentation, existing methods cannot separate multiple object instances. In this work, we address this limitation, and propose OpenMask3D, which is a zero-shot approach for open-vocabulary 3D instance segmentation. Guided by predicted class-agnostic 3D instance masks, our model aggregates per-mask features via multi-view fusion of CLIP-based image embeddings. Experiments and ablation studies on ScanNet200 and Replica show that OpenMask3D outperforms other open-vocabulary methods, especially on the long-tail distribution. Qualitative experiments further showcase OpenMask3D's ability to segment object properties based on free-form queries describing geometry, affordances, and materials.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
