面向可见光与长波红外图像块匹配的多域特征关系学习
Learning multi-domain feature relation for visible and Long-wave Infrared image patch matching
AI总结：
针对可见光与长波红外图像块匹配中数据集不足和尺度关系被忽略的问题，本文发布最大规模数据集VL-CMIM，并提出多域特征关系学习网络MD-FRN，通过空间与尺度关系学习及深层域交互机制提升跨光谱匹配鲁棒性。
摘要：
近年来，基于学习的算法在跨光谱图像块匹配上取得了令人鼓舞的性能，但距离实际应用仍远不能令人满意。一方面，缺乏包含多样场景的大规模数据集困扰着基于学习算法的进一步提升，因为其性能和泛化能力高度依赖数据集的规模与多样性。另一方面，现有研究更侧重空间域中的特征关系，而特征之间的尺度依赖关系常常被忽略，导致在跨光谱图像块出现显著外观变化时性能下降。为解决这些问题，我们据我们所知发布了最大的可见光与长波红外（LWIR）图像块匹配数据集，命名为VL-CMIM，其中包含1300对严格对齐的可见光与长波红外图像以及超过200万个图像块对，覆盖小行星、田野、乡村、建筑、街道和水域等多种场景。此外，我们提出了一种多域特征关系学习网络（MD-FRN）。该网络以四分支网络提取的特征作为输入，分别通过空间相关模块（SCM）和多尺度自适应聚合模块（MSAG）学习空间域和尺度域中的特征关系。为了进一步聚合多域关系，我们引入了一种深层域交互机制（DIM），在该机制中学习到的空间关系特征和尺度关系特征被交换，并进一步输入到MSCRM和SCM中。该机制使我们的模型能够学习交互式的跨域特征关系，从而对不同模态引起的显著外观变化具有更强的鲁棒性。
英文摘要：
Recently, learning-based algorithms have achieved promising performance on cross-spectral image patch matching, which, however, is still far from satisfactory for practical application. On the one hand, a lack of large-scale dataset with diverse scenes haunts its further improvement for learning-based algorithms, whose performances and generalization rely heavily on the dataset size and diversity. On the other hand, more emphasis has been put on feature relation in the spatial domain whereas the scale dependency between features has often been ignored, leading to performance degeneration especially when encountering significant appearance variations for cross-spectral patches. To address these issues, we publish, to be best of our knowledge, the largest visible and Long-wave Infrared (LWIR) image patch matching dataset, termed VL-CMIM, which contains 1300 pairs of strictly aligned visible and LWIR images and over 2 million patch pairs covering diverse scenes such as asteroid, field, country, build, street and water.In addition, a multi-domain feature relation learning network (MD-FRN) is proposed. Input by the features extracted from a four-branch network, both feature relations in spatial and scale domains are learned via a spatial correlation module (SCM) and multi-scale adaptive aggregation module (MSAG), respectively. To further aggregate the multi-domain relations, a deep domain interactive mechanism (DIM) is applied, where the learnt spatial-relation and scale-relation features are exchanged and further input into MSCRM and SCM. This mechanism allows our model to learn interactive cross-domain feature relations, leading to improved robustness to significant appearance changes due to different modality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
