面向未配准红外-可见光图像融合的双向自配准方法
Bi-directional Self-Registration for Misaligned Infrared-Visible Image Fusion
AI总结：
针对红外-可见光图像未对齐且配准缺乏真值的问题，提出自监督B-SR框架，通过PDG、IPDG和邻域动态对齐损失实现全局-局部一致配准，提升跨模态对齐与融合性能。
摘要：
获取精确对齐的多模态图像对是实现高质量多模态图像融合的基础。为解决当前多模态图像配准与融合方法缺乏真值的问题，本文提出一种新颖的自监督双向自配准框架（Bi-directional Self-Registration，B-SR）。具体而言，B-SR利用代理数据生成器（PDG）和逆代理数据生成器（IPDG）实现自监督全局-局部配准。存在空间未对齐差异的可见光-红外图像对通过配准模块进行对齐，以获得全局差异；同一图像对由PDG进行裁剪、翻转、拼接等处理后再对齐，以获得局部差异。IPDG将所得局部差异转换为伪全局差异，用于与全局差异执行全局-局部差异一致性约束。此外，为消除模态差异对配准模块的影响，本文设计了邻域动态对齐损失，以实现跨模态图像边缘对齐。在未对齐多模态图像上的大量实验表明，与同类竞争方法相比，所提方法在多模态图像对齐和融合方面均具有有效性。代码将公开发布。
英文摘要：
Acquiring accurately aligned multi-modal image pairs is fundamental for achieving high-quality multi-modal image fusion. To address the lack of ground truth in current multi-modal image registration and fusion methods, we propose a novel self-supervised \textbf{B}i-directional \textbf{S}elf-\textbf{R}egistration framework (\textbf{B-SR}). Specifically, B-SR utilizes a proxy data generator (PDG) and an inverse proxy data generator (IPDG) to achieve self-supervised global-local registration. Visible-infrared image pairs with spatially misaligned differences are aligned to obtain global differences through the registration module. The same image pairs are processed by PDG, such as cropping, flipping, stitching, etc., and then aligned to obtain local differences. IPDG converts the obtained local differences into pseudo-global differences, which are used to perform global-local difference consistency with the global differences. Furthermore, aiming at eliminating the effect of modal gaps on the registration module, we design a neighborhood dynamic alignment loss to achieve cross-modal image edge alignment. Extensive experiments on misaligned multi-modal images demonstrate the effectiveness of the proposed method in multi-modal image alignment and fusion against the competing methods. Our code will be publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
