Image matching, which aims to identify corresponding pixel locations between images, is crucial in a wide range of scientific disciplines, aiding in image registration, fusion, and analysis. In recent years, deep learning-based image matching algorithms have dramatically outperformed humans in rapidly and accurately finding large amounts of correspondences. However, when dealing with images captured under different imaging modalities that result in significant appearance changes, the performance of these algorithms often deteriorates due to the scarcity of annotated cross-modal training data. This limitation hinders applications in various fields that rely on multiple image modalities to obtain complementary information. To address this challenge, we propose a large-scale pre-training framework that utilizes synthetic cross-modal training signals, incorporating diverse data from various sources, to train models to recognize and match fundamental structures across images. This capability is transferable to real-world, unseen cross-modality image matching tasks. Our key finding is that the matching model trained with our framework achieves remarkable generalizability across more than eight unseen cross-modality registration tasks using the same network weight, substantially outperforming existing methods, whether designed for generalization or tailored for specific tasks. This advancement significantly enhances the applicability of image matching technologies across various scientific disciplines and paves the way for new applications in multi-modality human and artificial intelligence analysis and beyond.
Cross-Modality Feature Fusion Based on Structured State Space Duality for Multimodal Image Registration Network
Remote Sensing Image Processing and Fusion Group, School of Electronic Engineering, Xidian University（遥感图像处理与融合组，电子工程学院，西安电子科技大学）；National Key Laboratory of Radar Signal Processing, Xidian University（雷达信号处理国家级实验室，西安电子科技大学）
A 3D Cross-modal Keypoint Descriptor for MR-US Matching and Registration
Harvard Medical School（哈佛医学院）；Brigham and Women’s Hospital（布莱根妇女医院）；Inria（法国国家信息与自动化研究所）；Sorbonne Université（索邦大学）；Institut du Cerveau - Paris Brain Institute - ICM, CNRS, Inserm, AP-HP, Hôpital de la Pitié Salpêtrière（巴黎大脑研究所 - ICM, 法国国家科学研究中心, 法国国家健康与医学研究院, 巴黎公立医院集团, 皮提耶-萨勒佩特里医院）；Technical University of Munich（慕尼黑工业大学）
MIFNet: Learning Modality-Invariant Features for Generalizable Multimodal Image Matching
National Engineering Research Center for Multimedia Software（多媒体软件国家工程研究中心）；Institute of Artificial Intelligence（人工智能研究院）；School of Computer Science（计算机科学学院）；Hubei Key Laboratory of Multimedia and Network Communication Engineering（湖北省多媒体与网络通信工程重点实验室）；Lee Kong Chian School of Medicine（李光耀医学院）；Nanyang Technological University（南洋理工大学）；Ningbo Institute of Materials Technology and Engineering（宁波材料技术与工程研究所）；Chinese Academy of Sciences（中国科学院）；Institute for Infocomm Research (I 2 R)（信息与通信研究所以（I 2 R））；Agency for Science, Technology and Research (A*STAR)（科技研究局（A*STAR））
Grid-Reg: Detector-Free Gridized Feature Learning and Matching for Large-Scale SAR-Optical Image Registration
Shanghai Key Laboratory of Intelligent Sensing and Recognition（上海智能感知与识别重点实验室）；Shanghai Jiao Tong University（上海交通大学）；Center of Digital Innovation（数字创新中心）
Modality-Aware Feature Matching in Visual and Vision-Language Applications: A Comprehensive Survey
School of Computing and Artificial Intelligence, Jiangxi University of Finance and Economics（江西财经大学计算机与人工智能学院）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算机与数据科学学院）；School of Computer Science and Informatics, Cardiff University（卡迪夫大学计算机科学与信息学院）；School of Computing and Communications, Lancaster University（兰卡斯特大学计算机与通讯学院）；School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；Institute for Infocomm Research, Agency for Science, Technology and Research (A*STAR)（新加坡资讯研究院，科技研究局（A*STAR））；Department of Automation, Tsinghua University（清华大学自动化系）
Efficient Bilateral Cross-Modality Cluster Matching for Unsupervised Visible-Infrared Person ReID
Xidian University（西安电子科技大学）；Northwestern Polytechnical University（西北工业大学）；Zhejiang Lab（之江实验室）；Chongqing University of Posts and Telecommunications（重庆邮电大学）
Hyper-LLaVA: Hyperbolic Uncertainty-aware Modality-Balanced Routing for Multimodal Continual Instruction Tuning
Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机技术研究所）；Institute of Software Chinese Academy of Sciences（中国科学院软件研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Multi-Memory Matching for Unsupervised Visible-Infrared Person Re-Identification
Institute of Artificial Intelligence, Xiamen University（厦门大学人工智能研究院）；School of Informatics, Xiamen University（厦门大学信息学院）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；School of Computer Science and Technology, East China Normal University（华东师范大学计算机科学与技术学院）；Shanghai Key Laboratory of Computer Software Evaluating and Testing（上海市计算机软件评测重点实验室）
ModaFlow: Modality-Aware Flow Matching for High-Fidelity Virtual Try-On
South China University of Technology（华南理工大学）；Universitat de Barcelona（巴塞罗那大学）；Computer Vision Center（计算机视觉中心）；Guangdong Provincial Key Laboratory of Multimodal Big Data Intelligent Analysis（广东省多模态大数据智能分析重点实验室）
Mutual Information Guided Optimal Transport for Unsupervised Visible-Infrared Person Re-identification
School of Computer Science and Technology, East China Normal University（华东师范大学计算机科学与技术学院）；School of Informatics, Xiamen University（厦门大学信息学院）；Institute of Automation Chinese Academy of Sciences（中国科学院自动化研究所）；Nanjing University of Posts and Telecommunications（南京邮电大学）
BiXFormer: A Robust Framework for Maximizing Modality Effectiveness in Multi-Modal Semantic Segmentation
Graduate School of Informatics, Nagoya University（名古屋大学信息学研究科）；AI Thrust, The Hong Kong University of Science and Technology（香港科学与技术大学人工智能 thrust）；INSAIT, Sofia University（索菲亚大学INSAIT）
LISMARS；Hubei Luojia Laboratory（湖北珞珈实验室）；Wuhan University（武汉大学）；The University of Hong Kong（香港大学）；The Hong Kong Polytechnic University（香港理工大学）；Texas A&M University（德克萨斯农工大学）
Differentiable Registration of Images and LiDAR Point Clouds with VoxelPoint-to-Pixel Matching
Tsinghua University（清华大学）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；New York University Abu Dhabi（纽约大学阿布扎比分校）；Wayne State University（韦恩州立大学）
UAE: Universal Anatomical Embedding on Multi-modality Medical Images
National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, School of Computer Science and Engineering, Northwestern Polytechnical University（西北工业大学计算机科学与工程学院，空天地海一体化大数据应用技术国家工程实验室）；Department of Electronic Engineering, The Chinese University of Hong Kong（香港中文大学电子工程系）；Beijing United Family Hospital（北京和睦家医院）；The First Affiliated Hospital, Zhejiang University（浙江大学医学院附属第一医院）；Peking Union Medical College Hospital（北京协和医院）
Unsupervised Visible-Infrared ReID via Pseudo-label Correction and Modality-level Alignment
Artificial Intelligence Thrust, HKUST(GZ)（香港科技大学（广州）人工智能分支）；Center for AI Research (CAIR), University of Agder(UiA)（阿格德大学人工智能研究中心）；Dept. of Computer Science and Engineering, HKUST（香港科技大学计算机科学与工程系）
School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；School of Electrical and Information Engineering, The University of Sydney（悉尼大学电气与信息工程学院）
Compositional Cross-Modality Translation via Whole-Volume Multitask Latent Flow Matching
Università Campus Bio-Medico di Roma（罗马 Campus Bio-Medico 大学）；Umeå University（于默奥大学）；UniCamillus – Saint Camillus International University of Health Sciences（UniCamillus – 圣卡米勒斯国际健康科学大学）
