2D3D-MATR：用于图像与点云之间无检测配准的 2D-3D 匹配 Transformer
2D3D-MATR: 2D-3D Matching Transformer for Detection-free Registration between Images and Point Clouds
AI总结：
本文提出无检测的 2D3D-MATR，通过由粗到细的 Transformer 块匹配和多尺度金字塔实现图像与点云配准，性能显著优于 P2-Net。
摘要：
配准中普遍采用的“先检测后匹配”方法，在跨模态情形下因关键点检测不兼容和特征描述不一致而面临困难。我们提出 2D3D-MATR，这是一种用于图像与点云之间精确、鲁棒配准的无检测方法。该方法采用由粗到细的流程：首先计算输入图像与点云的下采样块之间的粗对应，然后将其扩展为像素与块区域内点之间的密集对应。粗级块匹配基于 Transformer，通过自注意力联合学习全局上下文约束，并通过交叉注意力学习跨模态相关性。为解决块匹配中的尺度歧义，我们为每个图像块构建多尺度金字塔，并学习为每个点块在适当分辨率层级找到最佳匹配的图像块。在两个公开基准上的大量实验表明，2D3D-MATR 在内点率上比此前最先进的 P2-Net 高出约 20 个百分点，在配准召回率上高出超过 10 个点。代码和模型可在 https://github.com/minhaolee/2D3DMATR 获取。
英文摘要：
The commonly adopted detect-then-match approach to registration finds difficulties in the cross-modality cases due to the incompatible keypoint detection and inconsistent feature description. We propose, 2D3D-MATR, a detection-free method for accurate and robust registration between images and point clouds. Our method adopts a coarse-to-fine pipeline where it first computes coarse correspondences between downsampled patches of the input image and the point cloud and then extends them to form dense correspondences between pixels and points within the patch region. The coarse-level patch matching is based on transformer which jointly learns global contextual constraints with self-attention and cross-modality correlations with cross-attention. To resolve the scale ambiguity in patch matching, we construct a multi-scale pyramid for each image patch and learn to find for each point patch the best matching image patch at a proper resolution level. Extensive experiments on two public benchmarks demonstrate that 2D3D-MATR outperforms the previous state-of-the-art P2-Net by around $20$ percentage points on inlier ratio and over $10$ points on registration recall. Our code and models are available at https://github.com/minhaolee/2D3DMATR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
