Diff$^2$I2P: 可微分的图像到点云配准与扩散先验
Diff$^2$I2P: Differentiable Image-to-Point Cloud Registration with Diffusion Prior
AI总结：
提出Diff$^2$I2P，利用扩散先验和可微分模块弥合图像与点云模态差距，提升跨模态配准性能，在7-Scenes基准上召回率提升超7%。
摘要：
学习跨模态对应关系对于图像到点云（I2P）配准至关重要。现有方法大多利用度量学习来强制跨模态特征对齐，忽视了图像和点数据之间固有的模态差距。因此，这种范式难以确保准确的跨模态对应关系。为此，受近期大型扩散模型在跨模态生成方面成功的启发，我们提出了Diff$^2$I2P，一个完全可微分的I2P配准框架，利用新颖且有效的扩散先验来弥合模态差距。具体而言，我们提出了一种控制侧分数蒸馏（CSD）技术，从深度条件扩散模型中蒸馏知识，直接优化预测的变换。然而，由于对应关系检索和PnP求解器的不可微性，变换上的梯度无法反向传播到跨模态特征上。为此，我们进一步提出了一种可变形对应调优（DCT）模块，以可微分的方式估计对应关系，随后使用可微分的PnP求解器进行变换估计。通过这两种设计，扩散模型作为强先验指导图像和点云的跨模态特征学习，以形成鲁棒的对应关系，从而显著提高了配准性能。大量实验结果表明，Diff$^2$I2P在7-Scenes基准上持续优于最先进的I2P配准方法，配准召回率提高了7%以上。
英文摘要：
Learning cross-modal correspondences is essential for image-to-point cloud (I2P) registration. Existing methods achieve this mostly by utilizing metric learning to enforce feature alignment across modalities, disregarding the inherent modality gap between image and point data. Consequently, this paradigm struggles to ensure accurate cross-modal correspondences. To this end, inspired by the cross-modal generation success of recent large diffusion models, we propose Diff$^2$I2P, a fully Differentiable I2P registration framework, leveraging a novel and effective Diffusion prior for bridging the modality gap. Specifically, we propose a Control-Side Score Distillation (CSD) technique to distill knowledge from a depth-conditioned diffusion model to directly optimize the predicted transformation. However, the gradients on the transformation fail to backpropagate onto the cross-modal features due to the non-differentiability of correspondence retrieval and PnP solver. To this end, we further propose a Deformable Correspondence Tuning (DCT) module to estimate the correspondences in a differentiable way, followed by the transformation estimation using a differentiable PnP solver. With these two designs, the Diffusion model serves as a strong prior to guide the cross-modal feature learning of image and point cloud for forming robust correspondences, which significantly improves the registration. Extensive experimental results demonstrate that Diff$^2$I2P consistently outperforms SoTA I2P registration methods, achieving over 7% improvement in registration recall on the 7-Scenes benchmark.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
