通过单阶段渐进式密集配准改进未对齐多模态图像融合
Improving Misaligned Multi-modality Image Fusion with One-stage Progressive Dense Registration
AI总结：
针对多模态图像未对齐导致的融合失真问题，提出单阶段跨模态多尺度渐进式密集配准方案，搭配融合子网络，提升未对齐跨模态图像的融合质量。
摘要：
多模态图像之间的未对齐问题给图像融合带来了挑战，具体表现为结构失真和边缘重影。现有研究通常采用先配准后融合的思路，一般使用两个级联的配准阶段，即粗配准和精配准，两个阶段都直接估计各自的目标形变场。本文认为，分离的两阶段配准结构不够紧凑，且直接估计目标形变场的精度不足。为解决这些挑战，我们提出了跨模态多尺度渐进式密集配准（Cross-modality Multi-scale Progressive Dense Registration, C-MPDR）方案，该方案仅通过单阶段优化即可完成从粗到精的配准，从而提升未对齐多模态图像的融合性能。具体而言，该方案包含两个核心组件：密集形变场融合（Deformation Field Fusion, DFF）模块和渐进式特征精修（Progressive Feature Fine, PFF）模块。DFF模块在当前尺度聚合预测得到的多尺度形变子场，PFF模块则逐步精修剩余的未对齐特征，二者协同工作以准确估计最终的形变场。此外，我们还构建了一个基于Transformer-Conv的融合（Transformer-Conv-based Fusion, TCF）子网络，该网络兼顾局部与长程特征依赖，能够从配准后的红外与可见光图像中捕获更具信息量的特征，以生成高质量的融合图像。大量实验分析表明，所提方法在未对齐跨模态图像融合任务中具有优越性能。
英文摘要：
Misalignments between multi-modality images pose challenges in image fusion, manifesting as structural distortions and edge ghosts. Existing efforts commonly resort to registering first and fusing later, typically employing two cascaded stages for registration,i.e., coarse registration and fine registration. Both stages directly estimate the respective target deformation fields. In this paper, we argue that the separated two-stage registration is not compact, and the direct estimation of the target deformation fields is not accurate enough. To address these challenges, we propose a Cross-modality Multi-scale Progressive Dense Registration (C-MPDR) scheme, which accomplishes the coarse-to-fine registration exclusively using a one-stage optimization, thus improving the fusion performance of misaligned multi-modality images. Specifically, two pivotal components are involved, a dense Deformation Field Fusion (DFF) module and a Progressive Feature Fine (PFF) module. The DFF aggregates the predicted multi-scale deformation sub-fields at the current scale, while the PFF progressively refines the remaining misaligned features. Both work together to accurately estimate the final deformation fields. In addition, we develop a Transformer-Conv-based Fusion (TCF) subnetwork that considers local and long-range feature dependencies, allowing us to capture more informative features from the registered infrared and visible images for the generation of high-quality fused images. Extensive experimental analysis demonstrates the superiority of the proposed method in the fusion of misaligned cross-modality images.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
