MV-DETR：基于多视角检测Transformer的多模态室内目标检测
MV-DETR: Multi-modality indoor object detection by Multi-View DEtecton TRansformers
AI总结：
针对RGBD深度数据预训练不足、3D视觉纹理难提取等问题，提出MV-DETR流水线，设计含视觉纹理与几何编码器的轻量VG模块，在ScanNetV2基准上取得78%AP的新SOTA。
摘要：
我们提出了一种新颖的MV-DETR流水线，这是一种兼具效果与效率的基于Transformer的检测方法。针对输入的RGBD数据，我们注意到RGB数据已有极强的预训练权重，而深度相关数据的预训练工作效果欠佳。首先，我们认为几何线索与纹理线索都至关重要，且可分别编码。其次，我们发现在3D空间中，视觉纹理特征相比几何特征更难提取。遗憾的是，仅有数千条数据的单个RGBD数据集不足以训练出用于视觉纹理特征提取的判别性滤波器。最后同样重要的是，我们设计了一个轻量级VG模块，由视觉纹理编码器、几何编码器和VG连接器组成。与V-DETR等先前的最优工作相比，预训练视觉编码器带来的性能提升清晰可见。在ScanNetV2数据集上开展的大量实验验证了我们方法的有效性。值得一提的是，我们的方法达到了78%的AP，在ScanNetv2基准上创下了新的最优纪录。
英文摘要：
We introduce a novel MV-DETR pipeline which is effective while efficient transformer based detection method. Given input RGBD data, we notice that there are super strong pretraining weights for RGB data while less effective works for depth related data. First and foremost , we argue that geometry and texture cues are both of vital importance while could be encoded separately. Secondly, we find that visual texture feature is relatively hard to extract compared with geometry feature in 3d space. Unfortunately, single RGBD dataset with thousands of data is not enough for training an discriminating filter for visual texture feature extraction. Last but certainly not the least, we designed a lightweight VG module consists of a visual textual encoder, a geometry encoder and a VG connector. Compared with previous state of the art works like V-DETR, gains from pretrained visual encoder could be seen. Extensive experiments on ScanNetV2 dataset shows the effectiveness of our method. It is worth mentioned that our method achieve 78\% AP which create new state of the art on ScanNetv2 benchmark.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
