arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2409.18431cs.CV

Search3D:层级开放词汇三维分割

Search3D: Hierarchical Open-Vocabulary 3D Segmentation

ETH Zurich(苏黎世联邦理工学院);Google Zurich(谷歌苏黎世分部);Google Munich(谷歌慕尼黑分部);Stanford University(斯坦福大学)

Ayca Takmaz, Alexandros Delitzas, Robert W. Sumner, Francis Engelmann, Johanna Wald, Federico Tombari

AI总结:

Search3D提出层级开放词汇三维场景表示,支持多粒度搜索,超越物体级查询,并在场景规模部件分割基准上优于基线,同时保持物体和材料分割性能。

摘要:

开放词汇三维分割使得利用自由形式的文本描述探索三维空间成为可能。现有的开放词汇三维实例分割方法主要聚焦于识别物体级别的实例,但在处理更细粒度的场景实体(如物体部件或由通用属性描述的区域)时存在困难。在本工作中,我们提出了Search3D,一种构建层级开放词汇三维场景表示的方法,支持在多个粒度级别进行三维搜索:细粒度物体部件、完整物体或由材料等属性描述的区域。与先前方法不同,Search3D转向一种更灵活的开放词汇三维搜索范式,超越了显式的以物体为中心的查询。为了系统评估,我们进一步基于MultiScan贡献了一个场景规模的开放词汇三维部件分割基准,并在ScanNet++上提供了一套开放词汇细粒度部件标注。Search3D在场景规模的开放词汇三维部件分割中优于基线方法,同时在分割三维物体和材料方面保持强劲性能。我们的项目页面为http://search3d-segmentation.github.io。

英文摘要:

Open-vocabulary 3D segmentation enables exploration of 3D spaces using free-form text descriptions. Existing methods for open-vocabulary 3D instance segmentation primarily focus on identifying object-level instances but struggle with finer-grained scene entities such as object parts, or regions described by generic attributes. In this work, we introduce Search3D, an approach to construct hierarchical open-vocabulary 3D scene representations, enabling 3D search at multiple levels of granularity: fine-grained object parts, entire objects, or regions described by attributes like materials. Unlike prior methods, Search3D shifts towards a more flexible open-vocabulary 3D search paradigm, moving beyond explicit object-centric queries. For systematic evaluation, we further contribute a scene-scale open-vocabulary 3D part segmentation benchmark based on MultiScan, along with a set of open-vocabulary fine-grained part annotations on ScanNet++. Search3D outperforms baselines in scene-scale open-vocabulary 3D part segmentation, while maintaining strong performance in segmenting 3D objects and materials. Our project page is http://search3d-segmentation.github.io.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑