Registration networks have shown great application potentials in medical image analysis. However, supervised training methods have a great demand for large and high-quality labeled datasets, which is time-consuming and sometimes impractical due to data sharing issues. Unsupervised image registration algorithms commonly employ intensity-based similarity measures as loss functions without any manual annotations. These methods estimate the parameterized transformations between pairs of moving and fixed images through the optimization of the network parameters during training. However, these methods become less effective when the image quality varies, e.g., some images are corrupted by substantial noise or artifacts. In this work, we propose a novel approach based on a low-rank representation, i.e., Regnet-LRR, to tackle the problem. We project noisy images into a noise-free low-rank space, and then compute the similarity between the images. Based on the low-rank similarity measure, we train the registration network to predict the dense deformation fields of noisy image pairs. We highlight that the low-rank projection is reformulated in a way that the registration network can successfully update gradients. With two tasks, i.e., cardiac and abdominal intra-modality registration, we demonstrate that the low-rank representation can boost the generalization ability and robustness of models as well as bring significant improvements in noisy data registration scenarios.
We propose a registration algorithm for 2D CT/MRI medical images with a new unsupervised end-to-end strategy using convolutional neural networks. The contributions of our algorithm are threefold: (1) We transplant traditional image registration algorithms to an end-to-end convolutional neural network framework, while maintaining the unsupervised nature of image registration problems. The image-to-image integrated framework can simultaneously learn both image features and transformation matrix for registration. (2) Training with additional data without any label can further improve the registra…
Medical Image Registration and Its Application in Retinal Images: A Review
Southern University of Science and Technology（南方科技大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Singapore Eye Research Institute（新加坡眼科研究所）
Medical image registration is a difficult problem. Not only a registration algorithm needs to capture both large and small scale image deformations, it also has to deal with global and local image intensity variations. In this paper we describe a new multiresolution elastic image registration method that challenges these difficulties in image registration. To capture large and small scale image deformations, we use both global and local affine transformation algorithms. To address global and local image intensity variations, we apply an image intensity standardization algorithm to correct imag…
We present a fast learning-based algorithm for deformable, pairwise 3D medical image registration. Current registration methods optimize an objective function independently for each pair of images, which can be time-consuming for large data. We define registration as a parametric function, and optimize its parameters given a set of images from a collection of interest. Given a new pair of scans, we can quickly compute a registration field by directly evaluating the function using the learned parameters. We model this function using a convolutional neural network (CNN), and use a spatial transf…
Uncertainty Estimation for Pretrained Medical Image Registration Models via Transformation Equivariance
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；The Hawkes Institute, University College London（霍克斯研究所，伦敦大学学院）；AI Laboratory, Massachusetts Institute of Technology（人工智能实验室，麻省理工学院）
Image registration is an essential but challenging task in medical image computing, especially for echocardiography, where the anatomical structures are relatively noisy compared to other imaging modalities. Traditional (non-learning) registration approaches rely on the iterative optimization of a similarity metric which is usually costly in time complexity. In recent years, convolutional neural network (CNN) based image registration methods have shown good effectiveness. In the meantime, recent studies show that the attention-based model (e.g., Transformer) can bring superior performance in p…
Conventional approaches to image registration consist of time consuming iterative methods. Most current deep learning (DL) based registration methods extract deep features to use in an iterative setting. We propose an end-to-end DL method for registering multimodal images. Our approach uses generative adversarial networks (GANs) that eliminates the need for time consuming iterative methods, and directly generates the registered image with the deformation field. Appropriate constraints in the GAN cost function produce accurately registered images in less than a second. Experiments demonstrate t…
The Hong Kong University of Science and Technology（香港科技大学）
Affine registration is indispensable in a comprehensive medical image registration pipeline. However, only a few studies focus on fast and robust affine registration algorithms. Most of these studies utilize convolutional neural networks (CNNs) to learn joint affine and non-parametric registration, while the standalone performance of the affine subnetwork is less explored. Moreover, existing CNN-based affine registration approaches focus either on the local misalignment or the global orientation and position of the input to predict the affine transformation matrix, which are sensitive to spati…
Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）；Beihang University（北京航空航天大学）；Microsoft Research（微软研究院）
3D medical image registration is of great clinical importance. However, supervised learning methods require a large amount of accurately annotated corresponding control points (or morphing), which are very difficult to obtain. Unsupervised learning methods ease the burden of manual annotation by exploiting unlabeled data without supervision. In this paper, we propose a new unsupervised learning method using convolutional neural networks under an end-to-end framework, Volume Tweening Network (VTN), for 3D medical image registration. We propose three innovative technical components: (1) An end-t…
We present a new unsupervised learning algorithm, "FAIM", for 3D medical image registration. With a different architecture than the popular "U-net", the network takes a pair of full image volumes and predicts the displacement fields needed to register source to target. Compared with "U-net" based registration networks such as VoxelMorph, FAIM has fewer trainable parameters but can achieve higher registration accuracy as judged by Dice score on region labels in the Mindboggle-101 dataset. Moreover, with the proposed penalty loss on negative Jacobian determinants, FAIM produces deformations with…
CentraleSupelec（中央理工-高等电力学院）；INRIA（法国国家信息与自动化研究所）；Universite Paris-Saclay（巴黎萨克雷大学）；Imperial College London（帝国理工学院）；TheraPanacea
During the last decades, the research community of medical imaging has witnessed continuous advances in image registration methods, which pushed the limits of the state-of-the-art and enabled the development of novel medical procedures. A particular type of image registration problem, known as slice-to-volume registration, played a fundamental role in areas like image guided surgeries and volumetric image reconstruction. However, to date, and despite the extensive literature available on this topic, no survey has been written to discuss this challenging problem. This paper introduces the first…
LLaMA-Reg: Using LLaMA 2 for Unsupervised Medical Image Registration
Faculty of Information Science Engineering and Automation, Kunming University of Science and Technology（昆明理工大学信息科学工程与自动化学院）；Taizhou Hospital of Zhejiang Province Affiliated with Wenzhou Medical University（温州医科大学附属浙江省台州医院）；Enze Hospital, Taizhou Enze Medical Centre (Group)（台州恩泽医疗中心（集团）恩泽医院）；The Second Affiliated Hospital of Dalian Medical University（大连医科大学附属第二医院）
In this work, we propose a self-supervised learning method for affine image registration on 3D medical images. Unlike optimisation-based methods, our affine image registration network (AIRNet) is designed to directly estimate the transformation parameters between two input images without using any metric, which represents the quality of the registration, as the optimising function. But since it is costly to manually identify the transformation parameters between any two images, we leverage the abundance of cheap unlabelled data to generate a synthetic dataset for the training of the model. Add…
DGIST（大邱庆北科学技术院）；Yeungnam University College of Medicine（岭南大学医学院）；Yeungnam University Hospital（岭南大学医院）
Non-rigid registration is a necessary but challenging task in medical imaging studies. Recently, unsupervised registration models have shown good performance, but they often require a large-scale training dataset and long training times. Therefore, in real world application where only dozens to hundreds of image pairs are available, existing models cannot be practically used. To address these limitations, we propose a novel unsupervised registration model which is integrated with a gradient-based meta learning framework. In particular, we train a meta learner which finds an initialization poin…
Deformable image registration is widely utilized in medical image analysis, but most proposed methods fail in the situation of complex deformations. In this paper, we pre-sent a cascaded feature warping network to perform the coarse-to-fine registration. To achieve this, a shared-weights encoder network is adopted to generate the feature pyramids for the unaligned images. The feature warping registration module is then used to estimate the deformation field at each level. The coarse-to-fine manner is implemented by cascading the module from the bottom level to the top level. Furthermore, the m…
MambaMorph: a Mamba-based Framework for Medical MR-CT Deformable Registration
Image Processing Center, Beihang University, Beijing, China（北京航空航天大学图像处理中心）；Robotics Institute, Beihang University, Beijing, China（北京航空航天大学机器人研究所）；Tongji Hospital, Huazhong University of Science and Technology, Wuhan, China（华中科技大学同济医院）
Robust Rigid and Non-Rigid Medical Image Registration Using Learnable Edge Kernels
Department of Mathematics, University of Innsbruck（因斯布鲁克大学数学系）；Department of Radiology, Medical University of Innsbruck（因斯布鲁克医学大学放射学系）；Neuroimaging Research Core Facility, Medical University of Innsbruck（因斯布鲁克医学大学神经影像研究核心设施）
General Vision Encoder Features as Guidance in Medical Image Registration
Klinikum Rechts der Isar, Technical University of Munich（慕尼黑工业大学右岸医院）；School of Computation, Information & Technology, Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）；Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich（亥姆霍兹慕尼黑中心生物医学影像机器学习研究所）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Cancer Research UK Cambridge Institute, University of Cambridge（剑桥大学英国癌症研究剑桥研究所）；Department of Oncology, University of Cambridge（剑桥大学肿瘤学系）；German Cancer Consortium (DKTK), Partner Site Munich（德国癌症联盟慕尼黑合作站点）；Department of Computing, Imperial College London（伦敦帝国理工学院计算机系）
In image registration, many efforts have been devoted to the development of alternatives to the popular normalized mutual information criterion. Concurrently to these efforts, an increasing number of works have demonstrated that substantial gains in registration accuracy can also be achieved by aligning structural representations of images rather than images themselves. Following this research path, we propose a new method for mono- and multimodal image registration based on the alignment of regularized vector fields derived from structural information such as gradient vector flow fields, a te…
uniGradICON: A Foundation Model for Medical Image Registration
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；University of Salzburg（萨尔茨堡大学）；University Paris-Est（巴黎东部大学）；Brigham and Women’s Hospital（布莱根妇女医院）；ÉTS Montréal（蒙特利尔工程学院）；Boston University（波士顿大学）
Acquisition-to-acquisition signal intensity variations (non-standardness) are inherent in MR images. Standardization is a post processing method for correcting inter-subject intensity variations through transforming all images from the given image gray scale into a standard gray scale wherein similar intensities achieve similar tissue meanings. The lack of a standard image intensity scale in MRI leads to many difficulties in tissue characterizability, image display, and analysis, including image segmentation. This phenomenon has been documented well; however, effects of standardization on medi…
Medical image registration and segmentation are critical tasks for several clinical procedures. Manual realisation of those tasks is time-consuming and the quality is highly dependent on the level of expertise of the physician. To mitigate that laborious task, automatic tools have been developed where the majority of solutions are supervised techniques. However, in medical domain, the strong assumption of having a well-representative ground truth is far from being realistic. To overcome this challenge, unsupervised techniques have been investigated. However, they are still limited in performan…
University of Freiburg（弗莱堡大学）；German Cancer Consortium (DKTK)（德国癌症联盟（DKTK））
Deformable image registration is a very important field of research in medical imaging. Recently multiple deep learning approaches were published in this area showing promising results. However, drawbacks of deep learning methods are the need for a large amount of training datasets and their inability to register unseen images different from the training datasets. One shot learning comes without the need of large training datasets and has already been proven to be applicable to 3D data. In this work we present a one shot registration approach for periodic motion tracking in 3D and 4D datasets.…
Universität zu Lübeck（吕贝克大学）；Institute of Medical Informatics（医学信息学研究所）
As in other areas of medical image analysis, e.g. semantic segmentation, deep learning is currently driving the development of new approaches for image registration. Multi-scale encoder-decoder network architectures achieve state-of-the-art accuracy on tasks such as intra-patient alignment of abdominal CT or brain MRI registration, especially when additional supervision, such as anatomical labels, is available. The success of these methods relies to a large extent on the outstanding ability of deep CNNs to extract descriptive visual features from the input images. In contrast to conventional m…
In this paper a fast triangular mesh based registration method is proposed. Having Template and Reference images as inputs, the template image is triangulated using a content adaptive mesh generation algorithm. Considering the pixel values at mesh nodes, interpolated using spline interpolation method for both of the images, the energy functional needed for image registration is minimized. The minimization process was achieved using a mesh based discretization of the distance measure and regularization term which resulted in a sparse system of linear equations, which due to the smaller size in …
Pyramid Attention Network for Medical Image Registration
School of Biomedical Engineering, Shenzhen University（深圳大学生物医学工程学院）；Medical UltraSound Image Computing (MUSIC) Lab（医学超声图像计算实验室）；Smart Medical Imaging, Learning and Engineering (SMILE) Lab（智能医学影像、学习与工程实验室）
Southern Medical University（南方医科大学）；Shenzhen University（深圳大学）
Deformation field estimation is an important and challenging issue in many medical image registration applications. In recent years, deep learning technique has become a promising approach for simplifying registration problems, and has been gradually applied to medical image registration. However, most existing deep learning registrations do not consider the problem that when the receptive field cannot cover the corresponding features in the moving image and the fixed image, it cannot output accurate displacement values. In fact, due to the limitation of the receptive field, the 3 x 3 kernel h…
Southeast University（东南大学）；Centre de Recherche en Information Biomédicale Sino-Français (CRIBs)（中法生物医学信息研究中心）；Univ Rennes（雷恩大学）；Inserm（法国国家健康与医学研究院）；LTSI - UMR1099（信号与图像处理实验室）
Deep learning-based medical image registration and segmentation joint models utilize the complementarity (augmentation data or weakly supervised data from registration, region constraints from segmentation) to bring mutual improvement in complex scene and few-shot situation. However, further adoption of the joint models are hindered: 1) the diversity of augmentation data is reduced limiting the further enhancement of segmentation, 2) misaligned regions in weakly supervised data disturb the training process, 3) lack of label-based region constraints in few-shot situation limits the registration…
Southeast University（东南大学）；Jiangsu Provincial Joint International Research Laboratory of Medical Information Processing（江苏省医学信息处理联合国际研究实验室）；University of Western Ontario（西安大略大学）
An effective backbone network is important to deep learning-based Deformable Medical Image Registration (DMIR), because it extracts and matches the features between two images to discover the mutual correspondence for fine registration. However, the existing deep networks focus on single image situation and are limited in registration task which is performed on paired images. Therefore, we advance a novel backbone network, XMorpher, for the effective corresponding feature representation in DMIR. 1) It proposes a novel full transformer architecture including dual parallel feature extraction net…
We present recursive cascaded networks, a general architecture that enables learning deep cascades, for deformable image registration. The proposed architecture is simple in design and can be built on any base network. The moving image is warped successively by each cascade and finally aligned to the fixed image; this procedure is recursive in a way that every cascade learns to perform a progressive deformation for the current warped image. The entire system is end-to-end and jointly trained in an unsupervised manner. In addition, enabled by the recursive architecture, one cascade can be itera…
Image registration plays an important role in medical image analysis. Conventional optimization based methods provide an accurate estimation due to the iterative process at the cost of expensive computation. Deep learning methods such as learn-to-map are much faster but either iterative or coarse-to-fine approach is required to improve accuracy for handling large motions. In this work, we proposed to learn a registration optimizer via a multi-scale neural ODE model. The inference consists of iterative gradient updates similar to a conventional gradient descent optimizer but in a much faster wa…
SAME++: A Self-supervised Anatomical eMbeddings Enhanced medical image registration framework using stable sampling and regularized transformation
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Alibaba Group（阿里巴巴集团）；Johns Hopkins University（约翰斯·霍普金斯大学）；The First Affiliated Hospital of Zhejiang University（浙江大学附属第一医院）
SpikeReg: Energy-Efficient 3D Deformable Medical Image Registration with Spiking Neural Networks
School of Electrical and Computer Engineering, College of Engineering, University of Tehran（德黑兰大学电气与计算机工程学院）；Max Planck Institute for Brain Research（马克斯·普朗克脑科学研究所）；School of Computer Engineering, Iran University of Science and Technology (IUST)（伊朗科学技术大学计算机工程学院）；Department of Electrical and Computer Engineering, University of Waterloo（滑铁卢大学电气与计算机工程系）
Computer Science and Artificial Intelligence Lab, MIT（麻省理工学院计算机科学与人工智能实验室）；Meinig School of Biomedical Engineering, Cornell University（康奈尔大学迈尼格生物医学工程学院）；Martinos Center for Biomedical Imaging, MGH, HMS（麻省总医院哈佛医学院马蒂诺生物医学成像中心）
We present VoxelMorph, a fast learning-based framework for deformable, pairwise medical image registration. Traditional registration methods optimize an objective function for each pair of images, which can be time-consuming for large datasets or rich deformation models. In contrast to this approach, and building on recent learning-based methods, we formulate registration as a function that maps an input image pair to a deformation field that aligns these images. We parameterize the function via a convolutional neural network (CNN), and optimize the parameters of the neural network on a set of…
On-the-Fly Guidance Training for Medical Image Registration
University of Leeds（利兹大学）；University of California, Irvine（加州大学欧文分校）；Tongji University（同济大学）；Huazhong University of Science and Technology（华中科技大学）；Fudan Univeristy（复旦大学）；University of California, San Diego（加州大学圣迭戈分校）
Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）；Cincinnati Children’s Hospital Medical Center（辛辛那提儿童医院医疗中心）；University of Cincinnati College of Medicine（辛辛那提大学医学院）；University of Cincinnati（辛辛那提大学）
Image registration is a critical component in the applications of various medical image analyses. In recent years, there has been a tremendous surge in the development of deep learning (DL)-based medical image registration models. This paper provides a comprehensive review of medical image registration. Firstly, a discussion is provided for supervised registration categories, for example, fully supervised, dual supervised, and weakly supervised registration. Next, similarity-based as well as generative adversarial network (GAN)-based registration are presented as part of unsupervised registrat…
University of Waterloo（滑铁卢大学）；Sunnybrook Research Institute（桑尼布鲁克研究所）
We present Recurrence with Correlation Network (RWCNet), a medical image registration network with multi-scale features and a cost volume layer. We demonstrate that these architectural features improve medical image registration accuracy in two image registration datasets prepared for the MICCAI 2022 Learn2Reg Workshop Challenge. On the large-displacement National Lung Screening Test (NLST) dataset, RWCNet is able to achieve a total registration error (TRE) of 2.11mm between corresponding keypoints without instance fine-tuning. On the OASIS brain MRI dataset, RWCNet is able to achieve an avera…
Institute of Mechatronic Systems, Leibniz Universität Hannover（汉诺威莱布尼茨大学机电系统研究所）
We present deformable unsupervised medical image registration using a randomly-initialized deep convolutional neural network (CNN) as regularization prior. Conventional registration methods predict a transformation by minimizing dissimilarities between an image pair. The minimization is usually regularized with manually engineered priors, which limits the potential of the registration. By learning transformation priors from a large dataset, CNNs have achieved great success in deformable registration. However, learned methods are restricted to domain-specific data and the required amounts of me…
Institute of Medical Informatics, Universität zu Lübeck（吕贝克大学医学信息学研究所）
Though, deep learning based medical image registration is currently starting to show promising advances, often, it still fells behind conventional frameworks in terms of registration accuracy. This is especially true for applications where large deformations exist, such as registration of interpatient abdominal MRI or inhale-to-exhale CT lung registration. Most current works use U-Net-like architectures to predict dense displacement fields from the input images in different supervised and unsupervised settings. We believe that the U-Net architecture itself to some level limits the ability to p…
College of Physics and Information Engineering, Fuzhou University（福州大学物理与信息工程学院）；School of Data Science, Fudan University（复旦大学大数据学院）；School of Biomedical Engineering, Shanghai Jiao Tong University（上海交通大学生物医学工程学院）
Multi-modality medical images can provide relevant or complementary information for a target (organ, tumor or tissue). Registering multi-modality images to a common space can fuse these comprehensive information, and bring convenience for clinical application. Recently, neural networks have been widely investigated to boost registration methods. However, it is still challenging to develop a multi-modality registration network due to the lack of robust criteria for network training. In this work, we propose a multi-modality registration network (MMRegNet), which can perform registration between…
Institute for Parallel and Distributed Systems, University of Stuttgart（斯图加特大学并行与分布式系统研究所）
Image registration is a fundamental task for medical imaging. Resampling of the intensity values is required during registration and better spatial resolution with finer and sharper structures can improve the resampling performance and hence the registration accuracy. Super-resolution (SR) is an algorithmic technique targeting at spatial resolution enhancement which can achieve an image resolution beyond the hardware limitation. In this work, we consider SR as a preprocessing technique and present a CNN-based resolution enhancement module (REM) which can be easily plugged into the registration…
Johns Hopkins Medical Institutes（约翰·霍普金斯医学院）；Johns Hopkins University（约翰·霍普金斯大学）；NVIDIA Corporation（英伟达公司）；Duke University Medical Center（杜克大学医学中心）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
In the last decade, convolutional neural networks (ConvNets) have been a major focus of research in medical image analysis. However, the performances of ConvNets may be limited by a lack of explicit consideration of the long-range spatial relationships in an image. Recently Vision Transformer architectures have been proposed to address the shortcomings of ConvNets and have produced state-of-the-art performances in many medical imaging applications. Transformers may be a strong candidate for image registration because their substantially larger receptive field enables a more precise comprehensi…
Institute of Parallel and Distributed Systems, University of Stuttgart（斯图加特大学并行与分布式系统研究所）
Deformable image registration is a fundamental task in medical imaging. Due to the large computational complexity of deformable registration of volumetric images, conventional iterative methods usually face the tradeoff between the registration accuracy and the computation time in practice. In order to boost the registration performance in both accuracy and runtime, we propose a fast convolutional neural network. Specially, to efficiently utilize the memory resources and enlarge the model capacity, we adopt additive forwarding instead of channel concatenation and deepen the network in each enc…
Registration plays an important role in medical image analysis. Deep learning-based methods have been studied for medical image registration, which leverage convolutional neural networks (CNNs) for efficiently regressing a dense deformation field from a pair of images. However, CNNs are limited in its ability to extract semantically meaningful intra- and inter-image spatial correspondences, which are of importance for accurate image registration. This study proposes a novel end-to-end deep learning-based framework for unsupervised affine and diffeomorphic deformable registration, referred as A…
Research institute for signals, systems and computational intelligence, sinc(i)（信号、系统与计算智能研究所（sinc(i)））；FICH-UNL（国立罗萨里奥大学医学与生物化学研究所）；CONICET（阿根廷国家科学与技术研究委员会）
Deformable image registration is a fundamental problem in the field of medical image analysis. During the last years, we have witnessed the advent of deep learning-based image registration methods which achieve state-of-the-art performance, and drastically reduce the required computational time. However, little work has been done regarding how can we encourage our models to produce not only accurate, but also anatomically plausible results, which is still an open question in the field. In this work, we argue that incorporating anatomical priors in the form of global constraints into the learni…
AdamFlow: Adam-based Wasserstein Gradient Flows for Surface Registration in Medical Imaging
Department of Brain Sciences, Imperial College London（帝国理工学院脑科学系）；Department of Computing, Imperial College London（帝国理工学院计算系）；School of Computer Science, University of Birmingham（伯明翰大学计算机科学学院）；Department of Computer Science, Columbia University（哥伦比亚大学计算机科学系）；Data Science Institute, Imperial College London（帝国理工学院数据科学研究所）
