只需添加几何信息：无需人类参与的免梯度开放词汇3D检测
Just Add Geometry: Gradient-Free Open-Vocabulary 3D Detection Without Human-in-the-Loop
AI总结：
本文提出一种免训练、开放词汇的3D目标检测方法，利用2D视觉-语言模型生成候选区域，结合SAM分割、相机几何和DBSCAN聚类推断3D边界框，并构建雾增强数据集Pseudo-nuScenes验证其在LiDAR和RGB-D输入下的竞争力。
摘要：
现代3D目标检测数据集受限于狭窄的类别分类体系和昂贵的人工标注，这限制了它们扩展到开放世界场景的能力。相比之下，基于网络规模图像-文本对训练的2D视觉-语言模型展现出丰富的语义理解能力，并支持通过自然语言提示进行开放词汇检测。在本工作中，我们利用2D基础模型的成熟度和类别多样性，在没有任何人工标注的3D标签的情况下执行开放词汇3D目标检测。我们的流程使用2D视觉-语言检测器生成以文本为条件的候选区域，这些候选区域通过SAM进行分割，并利用相机几何以及LiDAR或单目伪深度反投影到3D空间。我们引入了一种基于DBSCAN聚类和Rotating Calipers的几何膨胀策略，无需训练即可推断3D边界框。为了模拟不利的真实世界条件，我们构建了Pseudo-nuScenes，这是nuScenes数据集的一个雾增强、仅RGB变体。实验表明，我们的方法在多种设置下（包括基于LiDAR和纯RGB-D输入）均取得了有竞争力的定位性能，同时保持免训练和开放词汇的特性。我们的结果凸显了2D基础模型在可扩展3D感知方面尚未开发的潜力。我们在https://github.com/atharv0goel/open-world-3D-det开源了代码和资源。
英文摘要：
Modern 3D object detection datasets are constrained by narrow class taxonomies and costly manual annotations, limiting their ability to scale to open-world settings. In contrast, 2D vision-language models trained on web-scale image-text pairs exhibit rich semantic understanding and support open-vocabulary detection via natural language prompts. In this work, we leverage the maturity and category diversity of 2D foundation models to perform open-vocabulary 3D object detection without any human-annotated 3D labels. Our pipeline uses a 2D vision-language detector to generate text-conditioned proposals, which are segmented with SAM and back-projected into 3D using camera geometry and either LiDAR or monocular pseudo-depth. We introduce a geometric inflation strategy based on DBSCAN clustering and Rotating Calipers to infer 3D bounding boxes without training. To simulate adverse real-world conditions, we construct Pseudo-nuScenes, a fog-augmented, RGB-only variant of the nuScenes dataset. Experiments demonstrate that our method achieves competitive localization performance across multiple settings, including LiDAR-based and purely RGB-D inputs, all while remaining training-free and open-vocabulary. Our results highlight the untapped potential of 2D foundation models for scalable 3D perception. We open-source our code and resources at https://github.com/atharv0goel/open-world-3D-det.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
