开放词汇单目3D物体检测
Open Vocabulary Monocular 3D Object Detection
AI总结：
本文提出了一种开放词汇单目3D检测方法，通过整合预训练的2D和3D视觉模型，解决3D标注稀缺和语义歧义问题，实现对新类别的零样本检测和已有类别的域内检测。
摘要：
我们提出了并研究了开放词汇单目3D检测，这是一个新的任务，旨在从单个RGB图像中检测任意类别的物体在度量3D空间中的位置。现有的3D物体检测器要么依赖于成本较高的传感器如激光雷达或多视角设置，要么仍然局限于封闭词汇设置，只能检测有限的类别，限制了其应用范围。我们识别出这一新设置中的两个关键挑战。首先，3D包围盒注释的稀缺性限制了训练通用模型的能力。为了减少对3D监督的依赖，我们提出了一种框架，能够有效整合预训练的2D和3D视觉基础模型。其次，现有数据集中缺失标签和语义歧义（例如，桌子与书桌）阻碍了可靠的评估。为了解决这个问题，我们设计了一个新的度量标准，能够捕捉模型性能并缓解标注问题。我们的方法在零样本检测新类别以及在域内检测已见类别上均取得了最先进的结果。我们希望我们的方法提供了一个强大的基线，我们的评估协议为未来研究建立了一个可靠的基准。
英文摘要：
We propose and study open-vocabulary monocular 3D detection, a novel task that aims to detect objects of any categores in metric 3D space from a single RGB image. Existing 3D object detectors either rely on costly sensors such as LiDAR or multi-view setups, or remain confined to closed vocabularies settings with limited categories, restricting their applicability. We identify two key challenges in this new setting. First, the scarcity of 3D bounding box annotations limits the ability to train generalizable models. To reduce dependence on 3D supervision, we propose a framework that effectively integrates pretrained 2D and 3D vision foundation models. Second, missing labels and semantic ambiguities (\eg, table vs. desk) in existing datasets hinder reliable evaluation. To address this, we design a novel metric that captures model performance while mitigating annotation issues. Our approach achieves state-of-the-art results in zero-shot 3D detection of novel categories as well as in-domain detection on seen classes. We hope our method provides a strong baseline and our evaluation protocol establishes a reliable benchmark for future research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
