Open3DIS：利用2D掩码引导的开放词汇3D实例分割
Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidance
AI总结：
提出 Open3DIS，通过跨帧聚合2D实例掩码并映射为几何一致的3D物体提议，与3D类别无关提议融合，提升小尺度及模糊物体的开放词汇分割性能。
摘要：
我们提出 Open3DIS，这是一种旨在解决3D场景中开放词汇实例分割问题的新颖方案。3D环境中的物体呈现出多样的形状、尺度和颜色，使得精确的实例级识别成为一项具有挑战性的任务。近期开放词汇场景理解的进展通过采用类别无关的3D实例提议网络进行物体定位，并为每个3D掩码学习可查询特征，在该领域取得了显著进展。尽管这些方法能够生成高质量的实例提议，但它们在识别小尺度和几何上模糊的物体时存在困难。我们方法的核心思想是一个新模块，该模块跨帧聚合2D实例掩码，并将其映射到几何一致的点云区域，作为解决上述局限的高质量物体提议。随后，这些提议与3D类别无关实例提议相结合，以涵盖现实世界中广泛的物体。为验证我们的方法，我们在三个主流数据集上进行了实验，包括 ScanNet200、S3DIS 和 Replica，结果表明在分割不同类别物体方面，相较最先进方法取得了显著性能提升。
英文摘要：
We introduce Open3DIS, a novel solution designed to tackle the problem of Open-Vocabulary Instance Segmentation within 3D scenes. Objects within 3D environments exhibit diverse shapes, scales, and colors, making precise instance-level identification a challenging task. Recent advancements in Open-Vocabulary scene understanding have made significant strides in this area by employing class-agnostic 3D instance proposal networks for object localization and learning queryable features for each 3D mask. While these methods produce high-quality instance proposals, they struggle with identifying small-scale and geometrically ambiguous objects. The key idea of our method is a new module that aggregates 2D instance masks across frames and maps them to geometrically coherent point cloud regions as high-quality object proposals addressing the above limitations. These are then combined with 3D class-agnostic instance proposals to include a wide range of objects in the real world. To validate our approach, we conducted experiments on three prominent datasets, including ScanNet200, S3DIS, and Replica, demonstrating significant performance gains in segmenting objects with diverse categories over the state-of-the-art approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
