迈向RGB-NIR跨模态图像配准及超越
Towards RGB-NIR Cross-modality Image Registration and Beyond
AI总结：
本文提出RGB-NIR-IRegis基准用于评估RGB-NIR跨模态图像配准，并设计SGFormer基线方法利用高级语义指导减轻局部不一致特征的负面影响，实验验证了其有效性。
摘要：
本文聚焦于RGB（可见光）-NIR（近红外）跨模态图像配准领域，这对于许多下游视觉任务充分利用可见光和红外图像中的互补信息至关重要。在该领域，研究人员面临两个主要挑战：缺乏带有视角变化的正确标注基准来评估RGB-NIR跨模态配准方法，以及RGB-NIR跨模态图像间外观差异导致的不一致局部特征问题。为应对这些挑战，我们首先提出了RGB-NIR图像配准（RGB-NIR-IRegis）基准，首次实现了对RGB-NIR跨模态图像配准任务的公平全面评估。对先前方法的评估突显了RGB-NIR-IRegis基准带来的重大挑战，尤其是在具有视角变化的RGB-NIR图像对上。为分析性能不佳的原因，我们设计了若干指标，揭示可见光与红外图像间不一致局部特征对模型性能的负面影响。这进一步促使我们开发了一种名为语义引导Transformer（SGFormer）的基线方法，利用高级语义指导来减轻局部不一致特征的负面影响。尽管动机简单，大量实验结果表明了我们方法的有效性。
英文摘要：
This paper focuses on the area of RGB(visible)-NIR(near-infrared) cross-modality image registration, which is crucial for many downstream vision tasks to fully leverage the complementary information present in visible and infrared images. In this field, researchers face two primary challenges - the absence of a correctly-annotated benchmark with viewpoint variations for evaluating RGB-NIR cross-modality registration methods and the problem of inconsistent local features caused by the appearance discrepancy between RGB-NIR cross-modality images. To address these challenges, we first present the RGB-NIR Image Registration (RGB-NIR-IRegis) benchmark, which, for the first time, enables fair and comprehensive evaluations for the task of RGB-NIR cross-modality image registration. Evaluations of previous methods highlight the significant challenges posed by our RGB-NIR-IRegis benchmark, especially on RGB-NIR image pairs with viewpoint variations. To analyze the causes of the unsatisfying performance, we then design several metrics to reveal the toxic impact of inconsistent local features between visible and infrared images on the model performance. This further motivates us to develop a baseline method named Semantic Guidance Transformer (SGFormer), which utilizes high-level semantic guidance to mitigate the negative impact of local inconsistent features. Despite the simplicity of our motivation, extensive experimental results show the effectiveness of our method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
