Find n' Propagate：城市场景中的开放词汇3D目标检测
Find n' Propagate: Open-Vocabulary 3D Object Detection in Urban Environments
AI总结：
针对基于LiDAR的3D目标检测词汇受限、标注成本高的问题，提出通用Find n' Propagate方法，通过贪心框搜索、远程模拟器等设计，显著提升城市场景开放词汇3D检测的新类召回率与平均精度。
摘要：
在本研究中，我们解决了当前基于LiDAR的3D目标检测系统的局限性——这类系统受限于类别词汇量狭窄，且标注新目标类别的成本高昂。我们探索城市场景中的开放词汇（OV）学习，旨在利用预训练视觉语言模型（VLM）结合多传感器数据捕获新类目标实例。我们设计并基准测试了四种潜在的基线解决方案，根据其输入数据策略将其分为自顶向下或自底向上两类方法。这些方法虽然有效，但存在一定局限性，例如3D框估计时遗漏新类目标，或是应用了严格的先验知识，导致对靠近相机的目标或矩形几何形状的目标产生偏向。为克服这些局限，我们提出了一种面向3D开放词汇任务的通用\textsc{Find n' Propagate}方法，旨在最大化新类目标的召回率，并将这种检测能力传播到更远的区域，从而逐步捕获更多目标。具体而言，我们利用贪心框搜索器，在每个生成的视锥体中针对不同朝向和深度的3D新类框进行搜索，并通过交叉对齐和密度排序器确保新识别框的可靠性。此外，我们提出的远程模拟器通过在自训练过程中随机多样化伪标记的新类实例，结合记忆库中基础样本的融合，缓解了对靠近相机目标的固有偏向。大量实验表明，在不同的开放词汇设置、VLM和3D检测器下，新类目标召回率提升了53%。值得注意的是，新类目标的平均精度（AP）最高提升了3.97倍。源代码已发布于https://github.com/djamahl99/findnpropagate。
英文摘要：
In this work, we tackle the limitations of current LiDAR-based 3D object detection systems, which are hindered by a restricted class vocabulary and the high costs associated with annotating new object classes. Our exploration of open-vocabulary (OV) learning in urban environments aims to capture novel instances using pre-trained vision-language models (VLMs) with multi-sensor data. We design and benchmark a set of four potential solutions as baselines, categorizing them into either top-down or bottom-up approaches based on their input data strategies. While effective, these methods exhibit certain limitations, such as missing novel objects in 3D box estimation or applying rigorous priors, leading to biases towards objects near the camera or of rectangular geometries. To overcome these limitations, we introduce a universal \textsc{Find n' Propagate} approach for 3D OV tasks, aimed at maximizing the recall of novel objects and propagating this detection capability to more distant areas thereby progressively capturing more. In particular, we utilize a greedy box seeker to search against 3D novel boxes of varying orientations and depth in each generated frustum and ensure the reliability of newly identified boxes by cross alignment and density ranker. Additionally, the inherent bias towards camera-proximal objects is alleviated by the proposed remote simulator, which randomly diversifies pseudo-labeled novel instances in the self-training process, combined with the fusion of base samples in the memory bank. Extensive experiments demonstrate a 53% improvement in novel recall across diverse OV settings, VLMs, and 3D detectors. Notably, we achieve up to a 3.97-fold increase in Average Precision (AP) for novel object classes. The source code is made available at https://github.com/djamahl99/findnpropagate.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
