面向图像到点云配准的自监督跨模态学习
Self-Supervised Cross-Modal Learning for Image-to-Point Cloud Registration
AI总结：
本文提出自监督框架CrossI2P，通过双路径对比学习融合2D纹理与3D结构，结合由粗到细配准和动态训练机制，在KITTI和nuScenes上显著超越现有图像到点云配准方法。
摘要：
桥接2D与3D传感器模态对于自主系统中的鲁棒感知至关重要。然而，图像到点云（I2P）配准仍然具有挑战性，原因在于纹理丰富但深度模糊的图像与稀疏但度量精确的点云之间存在语义-几何鸿沟，以及现有方法倾向于收敛到局部最优。为克服这些局限，我们提出CrossI2P，一个自监督框架，在单一端到端流程中统一跨模态学习与两阶段配准。首先，我们通过双路径对比学习学习一个几何-语义融合嵌入空间，实现无需标注的2D纹理与3D结构双向对齐。其次，我们采用由粗到细的配准范式：全局阶段通过联合模态内上下文与跨模态交互建模建立超点-超像素对应关系，随后进行几何约束的点级细化以实现精确配准。第三，我们采用带有梯度归一化的动态训练机制，以平衡特征对齐、对应关系细化和位姿估计的损失。大量实验表明，CrossI2P在KITTI Odometry基准上超越最先进方法23.7%，在nuScenes上超越37.9%，显著提升了精度和鲁棒性。
英文摘要：
Bridging 2D and 3D sensor modalities is critical for robust perception in autonomous systems. However, image-to-point cloud (I2P) registration remains challenging due to the semantic-geometric gap between texture-rich but depth-ambiguous images and sparse yet metrically precise point clouds, as well as the tendency of existing methods to converge to local optima. To overcome these limitations, we introduce CrossI2P, a self-supervised framework that unifies cross-modal learning and two-stage registration in a single end-to-end pipeline. First, we learn a geometric-semantic fused embedding space via dual-path contrastive learning, enabling annotation-free, bidirectional alignment of 2D textures and 3D structures. Second, we adopt a coarse-to-fine registration paradigm: a global stage establishes superpoint-superpixel correspondences through joint intra-modal context and cross-modal interaction modeling, followed by a geometry-constrained point-level refinement for precise registration. Third, we employ a dynamic training mechanism with gradient normalization to balance losses for feature alignment, correspondence refinement, and pose estimation. Extensive experiments demonstrate that CrossI2P outperforms state-of-the-art methods by 23.7% on the KITTI Odometry benchmark and by 37.9% on nuScenes, significantly improving both accuracy and robustness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
