We introduce, XoFTR, a cross-modal cross-view method for local feature matching between thermal infrared (TIR) and visible images. Unlike visible images, TIR images are less susceptible to adverse lighting and weather conditions but present difficulties in matching due to significant texture and intensity differences. Current hand-crafted and learning-based methods for visible-TIR matching fall short in handling viewpoint, scale, and texture diversities. To address this, XoFTR incorporates masked image modeling pre-training and fine-tuning with pseudo-thermal image augmentation to handle the modality differences. Additionally, we introduce a refined matching pipeline that adjusts for scale discrepancies and enhances match reliability through sub-pixel level refinement. To validate our approach, we collect a comprehensive visible-thermal dataset, and show that our method outperforms existing methods on many benchmarks.
Universal Weighting Metric Learning for Cross-Modal Matching
Center for Future Media（未来媒体中心）；School of Computer Science and Engineering（计算机科学与工程学院）；University of Electronic Science and Technology of China（电子科技大学）
CM-Bench: A Comprehensive Cross-Modal Feature Matching Benchmark Bridging Visible and Infrared Images
School of Instrument Science and Opto-Electronics Engineering, Beijing Information Science and Technology University（北京信息科技大学仪器科学与光电工程学院）；Department of Precision Instrument, Tsinghua University（清华大学精密仪器系）
Modality-Aware Feature Matching in Visual and Vision-Language Applications: A Comprehensive Survey
School of Computing and Artificial Intelligence, Jiangxi University of Finance and Economics（江西财经大学计算机与人工智能学院）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算机与数据科学学院）；School of Computer Science and Informatics, Cardiff University（卡迪夫大学计算机科学与信息学院）；School of Computing and Communications, Lancaster University（兰卡斯特大学计算机与通讯学院）；School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；Institute for Infocomm Research, Agency for Science, Technology and Research (A*STAR)（新加坡资讯研究院，科技研究局（A*STAR））；Department of Automation, Tsinghua University（清华大学自动化系）
MIFNet: Learning Modality-Invariant Features for Generalizable Multimodal Image Matching
National Engineering Research Center for Multimedia Software（多媒体软件国家工程研究中心）；Institute of Artificial Intelligence（人工智能研究院）；School of Computer Science（计算机科学学院）；Hubei Key Laboratory of Multimedia and Network Communication Engineering（湖北省多媒体与网络通信工程重点实验室）；Lee Kong Chian School of Medicine（李光耀医学院）；Nanyang Technological University（南洋理工大学）；Ningbo Institute of Materials Technology and Engineering（宁波材料技术与工程研究所）；Chinese Academy of Sciences（中国科学院）；Institute for Infocomm Research (I 2 R)（信息与通信研究所以（I 2 R））；Agency for Science, Technology and Research (A*STAR)（科技研究局（A*STAR））
Few-Shot Referring Video Single- and Multi-Object Segmentation via Cross-Modal Affinity with Instance Sequence Matching
School of Computer Science and Technology, Anhui University of Technology（安徽工业大学计算机科学与技术学院）；Department of Computer Science and Engineering, Southern University of Science and Technology（南方科技大学计算机科学与工程系）；School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机与信息工程学院）
Sparse-to-dense Feature Matching: Intra and Inter domain Cross-modal Learning in Domain Adaptation for 3D Semantic Segmentation
Sichuan University（四川大学）；University of Electronic Science and Technology of China（电子科技大学）；Dalian University of Technology（大连理工大学）；National University of Defense Technology（国防科技大学）
School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；School of Electrical and Information Engineering, The University of Sydney（悉尼大学电气与信息工程学院）
WFM: 3D Wavelet Flow Matching for Ultrafast Multi-Modal MRI Synthesis
Department of Computer Science, Stanford University（斯坦福大学计算机科学系）；Machine and Hybrid Intelligence Lab, Feinberg School of Medicine, Northwestern University（北western大学费因伯格医学院机器与混合智能实验室）
本文提出WFM方法，通过学习基于小波空间条件均值的直接流，实现高效多模态MRI合成，单模型在BraTS 2024上达到26.8 dB PSNR和0.94 SSIM，速度比扩散模型快250-1000倍。
Multi-Memory Matching for Unsupervised Visible-Infrared Person Re-Identification
Institute of Artificial Intelligence, Xiamen University（厦门大学人工智能研究院）；School of Informatics, Xiamen University（厦门大学信息学院）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；School of Computer Science and Technology, East China Normal University（华东师范大学计算机科学与技术学院）；Shanghai Key Laboratory of Computer Software Evaluating and Testing（上海市计算机软件评测重点实验室）
UFM: Unified Feature Matching Pre-training with Multi-Modal Image Assistants
Information Science and Technology College, Dalian Maritime University（大连海事大学信息科学技术学院）；National Pilot School of Software, Yunnan University（云南大学国家示范性软件学院）；Yunnan Lanyi Network Technology Co（云南揽翼网络科技有限公司）
A 3D Cross-modal Keypoint Descriptor for MR-US Matching and Registration
Harvard Medical School（哈佛医学院）；Brigham and Women’s Hospital（布莱根妇女医院）；Inria（法国国家信息与自动化研究所）；Sorbonne Université（索邦大学）；Institut du Cerveau - Paris Brain Institute - ICM, CNRS, Inserm, AP-HP, Hôpital de la Pitié Salpêtrière（巴黎大脑研究所 - ICM, 法国国家科学研究中心, 法国国家健康与医学研究院, 巴黎公立医院集团, 皮提耶-萨勒佩特里医院）；Technical University of Munich（慕尼黑工业大学）
GraphAlign: Enhancing Accurate Feature Alignment by Graph matching for Multi-Modal 3D Object Detection
School of Computer and Information Technology, Beijing Jiaotong University（北京交通大学计算机与信息技术学院）；School of Information Science and Engineering, Hebei University of Science and Technology（河北科技大学信息科学与工程学院）；State Key Laboratory of Automotive Safety and Energy, Tsinghua University（清华大学汽车安全与节能国家重点实验室）
Efficient Bilateral Cross-Modality Cluster Matching for Unsupervised Visible-Infrared Person ReID
Xidian University（西安电子科技大学）；Northwestern Polytechnical University（西北工业大学）；Zhejiang Lab（之江实验室）；Chongqing University of Posts and Telecommunications（重庆邮电大学）
