OpenM3D：无需人工标注的开放词汇多视角室内3D目标检测
OpenM3D: Open Vocabulary Multi-view Indoor 3D Object Detection without Human Annotations
AI总结：
OpenM3D提出无需人工标注的开放词汇多视角室内3D目标检测方法，通过图嵌入生成高质量3D伪框并结合CLIP特征对齐训练单阶段检测器，在ScanNet200和ARKitScenes上以更快速度超越现有方法。
摘要：
开放词汇（OV）3D目标检测是一个新兴领域，但与基于3D点云的方法相比，通过基于图像的方法对其进行探索仍然有限。我们提出了OpenM3D，这是一种新颖的开放词汇多视角室内3D目标检测器，无需人工标注即可训练。具体而言，OpenM3D是一个单阶段检测器，适配了来自ImGeoNet模型的2D诱导体素特征。为了支持开放词汇，它联合使用类别无关的3D定位损失和体素-语义对齐损失进行训练，前者需要高质量的3D伪框，后者需要多样化的预训练CLIP特征。我们遵循OV-3DET的训练设置，即给定带位姿的RGB-D图像，但没有3D框或类别的人工标注。我们提出了一种使用图嵌入技术的3D伪框生成方法，将2D分割片段组合成连贯的3D结构。我们的伪框在精确率和召回率上均优于其他方法，包括OV-3DET中提出的方法。我们进一步从与每个连贯3D结构相关联的2D分割片段中采样多样化的CLIP特征，以与对应的体素特征对齐。训练高精度单阶段检测器的关键在于使两种损失都朝着高质量目标学习。在推理时，OpenM3D作为一种高效的检测器，仅需多视角图像作为输入，在ScanNet200和ARKitScenes室内基准上展现出优于现有方法的精度和速度（每场景0.3秒）。我们在精度和速度上均超越了一种强两阶段方法（该方法利用我们的类别无关检测器与基于ViT CLIP的开放词汇分类器）以及一种结合多视角深度估计器的基线方法。
英文摘要：
Open-vocabulary (OV) 3D object detection is an emerging field, yet its exploration through image-based methods remains limited compared to 3D point cloud-based methods. We introduce OpenM3D, a novel open-vocabulary multi-view indoor 3D object detector trained without human annotations. In particular, OpenM3D is a single-stage detector adapting the 2D-induced voxel features from the ImGeoNet model. To support OV, it is jointly trained with a class-agnostic 3D localization loss requiring high-quality 3D pseudo boxes and a voxel-semantic alignment loss requiring diverse pre-trained CLIP features. We follow the training setting of OV-3DET where posed RGB-D images are given but no human annotations of 3D boxes or classes are available. We propose a 3D Pseudo Box Generation method using a graph embedding technique that combines 2D segments into coherent 3D structures. Our pseudo-boxes achieve higher precision and recall than other methods, including the method proposed in OV-3DET. We further sample diverse CLIP features from 2D segments associated with each coherent 3D structure to align with the corresponding voxel feature. The key to training a highly accurate single-stage detector requires both losses to be learned toward high-quality targets. At inference, OpenM3D, a highly efficient detector, requires only multi-view images for input and demonstrates superior accuracy and speed (0.3 sec. per scene) on ScanNet200 and ARKitScenes indoor benchmarks compared to existing methods. We outperform a strong two-stage method that leverages our class-agnostic detector with a ViT CLIP-based OV classifier and a baseline incorporating multi-view depth estimator on both accuracy and speed.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
