arXiv 2403.117512025-04-19cs.CV

跨光谱图像块匹配的关系表示学习网络

Relational Representation Learning Network for Cross-Spectral Image Patch Matching

Chinese Academy of Sciences（中国科学院）；Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences（中国科学院机器人与智能制造创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Xidian University（西安电子科技大学）；The Chinese University of Hong Kong（香港中文大学）

Chuang Yu, Yunpeng Liu, Jinmiao Zhao, Dou Quan, Zelin Shi, Xiangyu Yue

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对跨光谱图像块匹配中忽略单个图像块内在特征的问题，提出关系表示学习网络RRL-Net，结合自编码器、特征交互学习、多维全局到局部注意力和多损失后剪枝策略，在多个公共数据集上取得最先进性能。

摘要：

近年来，特征关系学习在跨光谱图像块匹配中引起了广泛关注。然而，现有相关研究侧重于提取图像块特征之间的多种关系，却忽略了对单个图像块充分的内在特征表示。因此，我们提出了一种创新的关系表示学习思想，同时关注充分挖掘单个图像块的内在特征以及图像块特征之间的关系。基于此，我们构建了关系表示学习网络（RRL-Net）。具体而言，我们创新性地构建了一个自编码器来充分表征单个内在特征，并引入特征交互学习（FIL）模块来提取深层特征关系。为了进一步充分挖掘单个内在特征，我们构建了一个轻量级的多维全局到局部注意力（MGLA）模块，以增强单个图像块的全局特征提取，并捕获全局特征内的局部依赖。通过结合MGLA模块，我们进一步探索了特征提取网络，并构建了基于注意力的轻量级特征提取（ALFE）网络。此外，我们提出了一种多损失后剪枝（MLPP）优化策略，该策略在避免参数和推理时间增加的同时，极大地促进了网络优化。大量实验表明，我们的RRL-Net在多个公共数据集上取得了最先进的（SOTA）性能。我们的代码可在https://github.com/YuChuang1205/RRL-Net获取。

英文摘要：

Recently, feature relation learning has drawn widespread attention in cross-spectral image patch matching. However, existing related research focuses on extracting diverse relations between image patch features and ignores sufficient intrinsic feature representations of individual image patches. Therefore, we propose an innovative relational representation learning idea that simultaneously focuses on sufficiently mining the intrinsic features of individual image patches and the relations between image patch features. Based on this, we construct a Relational Representation Learning Network (RRL-Net). Specifically, we innovatively construct an autoencoder to fully characterize the individual intrinsic features, and introduce a feature interaction learning (FIL) module to extract deep-level feature relations. To further fully mine individual intrinsic features, a lightweight multi-dimensional global-to-local attention (MGLA) module is constructed to enhance the global feature extraction of individual image patches and capture local dependencies within global features. By combining the MGLA module, we further explore the feature extraction network and construct an attention-based lightweight feature extraction (ALFE) network. In addition, we propose a multi-loss post-pruning (MLPP) optimization strategy, which greatly promotes network optimization while avoiding increases in parameters and inference time. Extensive experiments demonstrate that our RRL-Net achieves state-of-the-art (SOTA) performance on multiple public datasets. Our code are available at https://github.com/YuChuang1205/RRL-Net.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。