辐射、旋转和尺度不变特征描述符用于多模态图像匹配
Radiation, Rotation and Scale Invariant Feature Descriptor for Multimodal Image Matching
浏览论文内容
中文总结 AI 辅助
提出RRSI特征描述符，通过双头区域采样和联合编码几何辐射关系，结合双向跨模态生成重建约束，实现多模态图像匹配，支持全旋转和四倍尺度变化。
中文摘要 AI 辅助
多模态图像匹配是多源信息融合的一项基本任务。然而，几何畸变和非线性辐射差异（NRD）严重限制了性能，尤其是在辐射、旋转和尺度变化的情况下。为解决这一问题，我们提出了一种辐射、旋转和尺度不变（RRSI）特征描述符。首先，双头区域采样（DHRS）模块在关键点邻域上同时进行笛卡尔和对数极坐标采样，在保留空间结构特性的同时增强对旋转和尺度变化的鲁棒性。然后，我们在统一的深度特征空间中联合编码多模态图像之间的几何和辐射关系，实现模态内、双头采样和模态间区域的特征编码、交互和融合。此外，我们在训练过程中引入了双向跨模态生成重建约束。通过将隐式特征解码为对应模态的结构补丁，该机制在不增加推理开销的情况下锚定了模态不变的几何拓扑。在光学-红外和光学-SAR数据集上的实验表明，该方法具有高度竞争力的匹配性能和较强的旋转和尺度变化鲁棒性。RRSI支持从0到360度的全旋转范围和最高四倍的尺度因子。其泛化能力在计算机视觉、遥感和医学成像的多模态图像上得到了进一步验证。该实现将在此https URL上公开发布。
英文摘要
Multimodal image matching is a fundamental task for multi-source information fusion. However, geometric distortions and nonlinear radiometric differences (NRD) severely limit performance, especially under radiometric, rotation, and scale variations. To address this issue, we propose a radiation, rotation, and scale invariant (RRSI) feature descriptor. First, a dual-head regional sampling (DHRS) module simultaneously performs Cartesian and Log-Polar sampling on keypoint neighborhoods, retaining spatial structural properties while enhancing robustness to rotation and scale variations. We then jointly encode geometric and radiometric relations between multimodal images in a unified deep feature space, enabling feature encoding, interaction, and fusion across intra-modal, dual-head sampled, and inter-modal regions. Furthermore, we introduce a bidirectional cross-modal generative reconstruction constraint during training. By decoding implicit features into structural patches of the counterpart modality, this mechanism anchors modality-invariant geometric topologies without additional inference overhead. Experiments on optical-infrared and optical-SAR datasets demonstrate highly competitive matching performance and strong robustness to rotation and scale variations. RRSI supports the full rotation range from 0 to 360 degrees and scale factors up to four. Its generalization ability is further validated on multimodal images from computer vision, remote sensing, and medical imaging. The implementation will be made publicly available at https://github.com/yeyuanxin110/RRSI .
发表机构
- Southwest Jiaotong University（西南交通大学）
- Wuhan University（武汉大学）
机构由 AI 辅助整理，请以论文原文为准。
