arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.17686cs.CV
收藏

迈向开放世界中的3D物体性学习

Towards 3D Objectness Learning in an Open World

Rutgers University(罗格斯大学);Tsinghua University(清华大学);Michigan State University(密歇根州立大学);Florida State University(佛罗里达州立大学)

Taichi Liu, Zhenyu Wang, Ruofeng Liu, Guang Wang, Desheng Zhang

AI总结:

本文提出无提示、类别无关的OP3Det,融合2D基础模型语义先验、3D几何先验及跨模态混合专家,以学习开放世界3D物体性并显著提升检测性能。

摘要:

近期,3D物体检测和新类别检测的进展取得了显著进步,但关于学习泛化3D物体性的研究仍然不足。本文深入研究开放世界3D物体性学习,其重点是检测3D场景中的所有物体,包括训练期间未见过的新物体。传统的闭集3D检测器难以泛化到开放世界场景,而直接引入3D开放词汇模型以获得开放世界能力,则会面临词汇扩展和语义重叠的困难。为实现泛化的3D物体发现,我们提出OP3Det,这是一种类别无关的开放世界无提示3D检测器,无需依赖手工制作的文本提示即可检测3D场景内的任何物体。我们引入2D基础模型的强泛化和零样本能力,同时利用2D语义先验和3D几何先验生成类别无关建议,以拓宽3D物体发现。随后,通过在跨模态混合专家中整合来自点云和RGB图像的互补信息,OP3Det动态路由单模态和多模态特征,以学习泛化的3D物体性。大量实验证明了OP3Det的卓越性能,其AR最高显著超越现有开放世界3D检测器16.0%,并且相比闭世界3D检测器实现13.5%的提升。

英文摘要:

Recent advancements in 3D object detection and novel category detection have made significant progress, yet research on learning generalized 3D objectness remains insufficient. In this paper, we delve into learning open-world 3D objectness, which focuses on detecting all objects in a 3D scene, including novel objects unseen during training. Traditional closed-set 3D detectors struggle to generalize to open-world scenarios, while directly incorporating 3D open-vocabulary models for open-world ability struggles with vocabulary expansion and semantic overlap. To achieve generalized 3D object discovery, We propose OP3Det, a class-agnostic Open-World Prompt-free 3D Detector to detect any objects within 3D scenes without relying on hand-crafted text prompts. We introduce the strong generalization and zero-shot capabilities of 2D foundation models, utilizing both 2D semantic priors and 3D geometric priors for class-agnostic proposals to broaden 3D object discovery. Then, by integrating complementary information from point cloud and RGB image in the cross-modal mixture of experts, OP3Det dynamically routes uni-modal and multi-modal features to learn generalized 3D objectness. Extensive experiments demonstrate the extraordinary performance of OP3Det, which significantly surpasses existing open-world 3D detectors by up to 16.0% in AR and achieves a 13.5% improvement compared to closed-world 3D detectors.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑