Cross-Modality Feature Fusion Based on Structured State Space Duality for Multimodal Image Registration Network
Remote Sensing Image Processing and Fusion Group, School of Electronic Engineering, Xidian University(遥感图像处理与融合组,电子工程学院,西安电子科技大学);National Key Laboratory of Radar Signal Processing, Xidian University(雷达信号处理国家级实验室,西安电子科技大学)
In multi-modal image registration, the primary challenge lies in shared structural information extraction. Compared to Transformers, Structured State Space Duality (SSD) offers greater global structural feature extraction with higher efficiency during training and inference. Inspired by these advantages, we propose a novel algorithm for multi-modal image registration, named RegNetMamba-2. Our algorithm incorporates SSD into coarse-to-fine matching process to extract local and global structural features effectively. Firstly, SSD is applied in three different scales for multi-modal feature extraction in our network. To strengthen local representation, we pay more attention on foreground edge and structural information by feature scaling function of SSD. Secondly, for shared feature extraction of input images and multi-modal feature fusion in all scales, we propose cross-modality feature fusion model based on SSD, consisting of Cross-Modality feature Interaction (CMI) module and Multi-Scale feature Fusion (MSF) module. CMI module is designed for cross-modality feature extraction of each scale by SSD in cross form. MSF module is designed to employ a progressive upward fusion in feature-level to obtain fine features, consisting of multi-modal features in all scales. Following coarse-to-fine, the features in 1/8 scale from CMI and 1/2 scale from MSF are collected to calculate matching probability scores. Then we respectively establish matching process by correspondences of pixel-wise. Extensive experiments demonstrate that comparing with state-of-the-art deep-learning based algorithms, RegNetMamba-2 has achieved good effects in both performance and efficiency for multi-modal image registration on the following datasets: VIS-SAR (OSDataset), VIS-IR (LGHD/RoadSence) and VIS-NIR (RGB-NIR sense).
Deep Learning based Inter-Modality Image Registration Supervised by Intra-Modality Similarity
Northwestern Polytechnical University(西北工业大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);Shanghai United Imaging Intelligence Co., Ltd(上海联影智能有限公司);Shanghai Jiao Tong University(上海交通大学)
FusionRegister: Every Infrared and Visible Image Fusion Deserves Registration
School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, China(江南大学人工智能与计算机科学学院);School of Electronic and Information Engineering, Suzhou University of Science and Technology, Suzhou, China(苏州科技大学电子与信息工程学院)
Disentangle-then-Align: Non-Iterative Hybrid Multimodal Image Registration via Cross-Scale Feature Disentanglement
Faculty of Engineering and IT, University of Technology Sydney(新南威尔士大学工程与信息技术学院);School of Artificial Intelligence, Anhui University(安徽大学人工智能学院);School of Computer Science and Technology, Anhui University(安徽大学计算机科学与技术学院)
multimodars: A Rust-powered toolkit for multi-modality cardiac image fusion and registration
Department of Cardiology, Inselspital, Bern University Hospital, University of Bern(伯尔尼大学伯尔尼大学医院 Inselspital 心脏病学系);Graduate School for Cellular and Biomedical Sciences, University of Bern(伯尔尼大学细胞与生物医学科学研究生院)
A 3D Cross-modal Keypoint Descriptor for MR-US Matching and Registration
Harvard Medical School(哈佛医学院);Brigham and Women’s Hospital(布莱根妇女医院);Inria(法国国家信息与自动化研究所);Sorbonne Université(索邦大学);Institut du Cerveau - Paris Brain Institute - ICM, CNRS, Inserm, AP-HP, Hôpital de la Pitié Salpêtrière(巴黎大脑研究所 - ICM, 法国国家科学研究中心, 法国国家健康与医学研究院, 巴黎公立医院集团, 皮提耶-萨勒佩特里医院);Technical University of Munich(慕尼黑工业大学)
Unsupervised MR-US Multimodal Image Registration with Multilevel Correlation Pyramidal Optimization
School of Artificial Intelligence and Robotics, Hunan University, Changsha, Hunan, China(人工智能与机器人学院,湖南大学,长沙,湖南,中国);National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, Changsha, Hunan, China(机器人视觉感知与控制技术国家工程研究中心,湖南大学,长沙,湖南,中国);National University of Singapore(新加坡国立大学);Cornell University(康奈尔大学)
Domain-Shift Immunity in Deep Deformable Registration via Local Feature Representations
Kahlert School of Computing(卡尔伯特计算学校);Scientific Computing and Imaging Institute(科学计算与成像研究所);University of Utah(犹他大学);Department of Biomedical Engineering(生物医学工程系)
Surrogate Supervision for Robust and Generalizable Deformable Image Registration
Department of Electrical and Computer Engineering, Vanderbilt University(维斯尼尔大学电气与计算机工程系);Department of Radiology and Radiological Science, Johns Hopkins Medical School(约翰霍普金斯医学学校放射学与放射科学系);Image Analysis and Communications Laboratory in the Department of Electrical and Computer Engineering, Johns Hopkins University(约翰霍普金斯大学电气与计算机工程系图像分析与通信实验室);Center for Cognitive Medicine, Department of Psychiatry and Behavioral Science, Vanderbilt University Medical Center(维斯尼尔大学医学中心认知医学中心);University of Pittsburgh, School of Medicine(匹兹堡大学医学院)
Vector Field Attention for Deformable Image Registration
Johns Hopkins University(约翰斯·霍普金斯大学);Johns Hopkins School of Medicine(约翰斯·霍普金斯医学院);National Institute on Aging(美国国家老龄化研究所);National Institute of Health(美国国立卫生研究院)
A Generative Approach for Image Registration of Visible-Thermal (VT) Cancer Faces
Booz Allen Hamilton(博思艾伦汉密尔顿公司);University of Maryland Baltimore County(马里兰大学巴尔的摩县分校);National Cancer Institute, National Institutes of Health(美国国立卫生研究院国家癌症研究所)
Grid-Reg: Detector-Free Gridized Feature Learning and Matching for Large-Scale SAR-Optical Image Registration
Shanghai Key Laboratory of Intelligent Sensing and Recognition(上海智能感知与识别重点实验室);Shanghai Jiao Tong University(上海交通大学);Center of Digital Innovation(数字创新中心)
TAR: Text Semantic Assisted Cross-modal Image Registration Framework for Optical and SAR Images
Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education of China, School of Artificial Intelligence, Xidian University(中国教育部智能感知与图像理解重点实验室,西安电子科技大学人工智能学院);Department of Automation, Tsinghua University(清华大学自动化系)
The Brain Resection Multimodal Image Registration (ReMIND2Reg) 2025 Challenge
Inria Saclay Île-de-France(法国里昂萨克利研究所);CEA(法国原子能委员会);Université Paris-Saclay(巴黎-萨克利大学);Sorbonne Université(索邦大学);Institut du Cerveau - Paris Brain Institute - ICM(巴黎脑研究所);CNRS(法国国家科学研究中心);Inria(法国国家信息与自动化技术研究所);Inserm(法国国家医学研究院);AP-HP(法国国家医院集团);Hôpital de la Pitié Salpêtrière(皮蒂埃-萨尔普里埃尔医院);Harvard Medical School(哈佛医学院);Brigham and Women's Hospital(布里根医院);Department of Radiology and Radiological Science(放射学与放射科学系);Johns Hopkins Medical School(约翰霍普金斯医学院);Institute of Medical Informatics(医学信息学研究所);University of Lübeck(吕贝克大学);Image Analysis and Communications Laboratory(图像分析与通信实验室);Department of Electrical and Computer Engineering(电气与计算机工程系);Johns Hopkins University(约翰霍普金斯大学);CSAIL(媒体实验室)
Modality-Agnostic Structural Image Representation Learning for Deformable Multi-Modality Medical Image Registration
DAMO Academy, Alibaba Group(阿里巴巴达摩院);Hupan Lab(湖畔实验室);College of Computer Science and Technology, Zhejiang University, China(浙江大学计算机科学与技术学院);Shengjing Hospital of China Medical University, China(中国医科大学附属盛京医院)
Unsupervised Multi-Modality Registration Network based on Spatially Encoded Gradient Information
College of Physics and Information Engineering, Fuzhou University(福州大学物理与信息工程学院);School of Data Science, Fudan University(复旦大学大数据学院);School of Biomedical Engineering, Shanghai Jiao Tong University(上海交通大学生物医学工程学院)
Synthesis and Inpainting-Based MR-CT Registration for Image-Guided Thermal Ablation of Liver Tumors
Institute for Medical Imaging Technology, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200030, China(中国上海200030,上海交通大学生物医学工程学院医学影像技术研究所);Department of Radiology and Biomedical Research Imaging Center (BRIC), University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA(美国北卡罗来纳州教堂山27599,北卡罗来纳大学教堂山分校放射学与生物医学研究成像中心(BRIC));North China Electric Power University, Beijing, China(中国北京华北电力大学);Shanghai United Imaging Intelligence Co., Ltd, Shanghai, China(中国上海上海联影智能科技有限公司);Shanghai Cancer Center, Fudan University, Shanghai, China(中国上海复旦大学附属肿瘤医院)
LISMARS;Hubei Luojia Laboratory(湖北珞珈实验室);Wuhan University(武汉大学);The University of Hong Kong(香港大学);The Hong Kong Polytechnic University(香港理工大学);Texas A&M University(德克萨斯农工大学)
Modality-agnostic, patient-specific digital twins modeling temporally varying digestive motion
Computer and Information Science, Cornell University(康奈尔大学计算机与信息科学系);Department of Medical Physics, Memorial Sloan Kettering Cancer Center(纪念斯隆凯特琳癌症中心医学物理系);University Medical Center Utrecht(乌得勒支大学医学中心);Department of Radiation Physics, University of Texas MD Anderson Cancer Center(德克萨斯大学MD安德森癌症中心放射物理系);Carl E. Ravin Advanced Imaging Laboratories and Center for Virtual Imaging Trials, Duke University Medical Center(杜克大学医学中心卡尔·E·拉文高级影像实验室和虚拟影像试验中心);Department of Radiation Oncology, University of Michigan(密歇根大学放射肿瘤学系)
MambaMorph: a Mamba-based Framework for Medical MR-CT Deformable Registration
Image Processing Center, Beihang University, Beijing, China(北京航空航天大学图像处理中心);Robotics Institute, Beihang University, Beijing, China(北京航空航天大学机器人研究所);Tongji Hospital, Huazhong University of Science and Technology, Wuhan, China(华中科技大学同济医院)