Our anatomy is in constant motion. With modern MR imaging it is possible to record this motion in real-time during an ongoing radiation therapy session. In this paper we present an image registration method that exploits the sequential nature of 2D MR images to estimate the corresponding displacement field. The method employs several discriminative correlation filters that independently track specific points. Together with a sparse-to-dense interpolation scheme we can then estimate of the displacement field. The discriminative correlation filters are trained online, and our method is modality agnostic. For the interpolation scheme we use a neural network with normalized convolutions that is trained using synthetic diffeomorphic displacement fields. The method is evaluated on a segmented cardiac dataset and when compared to two conventional methods we observe an improved performance. This improvement is especially pronounced when it comes to the detection of larger motions of small objects.
The Hong Kong University of Science and Technology（香港科技大学）
Registration of pre-operative and follow-up brain MRI scans is challenging due to the large variation of tissue appearance and missing correspondences in tumour recurrence regions caused by tumour mass effect. Although recent deep learning-based deformable registration methods have achieved remarkable success in various medical applications, most of them are not capable of registering images with pathologies. In this paper, we propose a 3-step registration pipeline for pre-operative and follow-up brain MRI scans that consists of 1) a multi-level affine registration, 2) a conditional deep Lapla…
Deep learning-based group-wise registration for longitudinal MRI analysis in glioma
Erasmus MC（伊拉斯姆斯大学医学中心）；Medical Delta（医疗三角洲）；Delft University of Technology（代尔夫特理工大学）；Amsterdam UMC / VU Medical Center（阿姆斯特丹大学医学中心/自由大学医学中心）；Princess Máxima Center for Pediatric Oncology（马克西玛公主儿童肿瘤中心）；The Jackson Laboratory for Genomic Medicine（杰克逊基因组医学实验室）
Biomedical Imaging Research Institute, Cedars-Sinai Medical Center, United States Berlin Ultrahigh Field Facility (B.U.F.F.), Max Delbrück Center for Molecular Medicine in the Helmholtz Association (MDC), Germany Department of Imaging Physics, Delft University of Technology, Netherlands TUM School of Computation, Information；Technology, Technische Universität München, Germany Department of Surgery, Taipei Veterans General Hospital, Taiwan
Multiparametric mapping MRI has become a viable tool for myocardial tissue characterization. However, misalignment between multiparametric maps makes pixel-wise analysis challenging. To address this challenge, we developed a generalizable physics-informed deep-learning model using test-time adaptation to enable group image registration across contrast weighted images acquired from multiple physical models (e.g., a T1 mapping model and T2 mapping model). The physics-informed adaptation utilized the synthetic images from specific physics model as registration reference, allows for transductive l…
ShanghaiTech University（上海科技大学）；University of Zürich（苏黎世大学）；ETH Zürich（苏黎世联邦理工学院）
Multi-spectral optoacoustic tomography (MSOT) is an emerging optical imaging method providing multiplex molecular and functional information from the rodent brain. It can be greatly augmented by magnetic resonance imaging (MRI) that offers excellent soft-tissue contrast and high-resolution brain anatomy. Nevertheless, registration of multi-modal images remains challenging, chiefly due to the entirely different image contrast rendered by these modalities. Previously reported registration algorithms mostly relied on manual user-dependent brain segmentation, which compromised data interpretation …
The Hong Kong University of Science and Technology（香港科技大学）
Registration of pre-operative and post-recurrence brain images is often needed to evaluate the effectiveness of brain gliomas treatment. While recent deep learning-based deformable registration methods have achieved remarkable success with healthy brain images, most of them would be unable to accurately align images with pathologies due to the absent correspondences in the reference image. In this paper, we propose a deep learning-based deformable registration method that jointly estimates regions with absent correspondence and bidirectional deformation fields. A forward-backward consistency c…
University of Edinburgh（爱丁堡大学）；Imperial College London（帝国理工学院）；University of Essex（埃塞克斯大学）
Magnetic Resonance Imaging (MRI) typically recruits multiple sequences (defined here as "modalities"). As each modality is designed to offer different anatomical and functional clinical information, there are evident disparities in the imaging content across modalities. Inter- and intra-modality affine and non-rigid image registration is an essential medical image analysis process in clinical imaging, as for example before imaging biomarkers need to be derived and clinically evaluated across different MRI modalities, time phases and slices. Although commonly needed in real clinical scenarios, …
University of North Carolina（北卡罗来纳大学）；Shanghai Jiao Tong University（上海交通大学）
Deformable image registration is fundamental to longitudinal and population analysis. Geometric alignment of the infant brain MR images is challenging, owing to rapid changes in image appearance in association with brain development. In this paper, we propose an infant-dedicated deep registration network that uses the auto-context strategy to gradually refine the deformation fields to obtain highly accurate correspondences. Instead of training multiple registration networks, our method estimates the deformation fields by invoking a single network multiple times for iterative deformation refine…
Deformable MRI Sequence Registration for AI-based Prostate Cancer Diagnosis
Radboudumc（拉德堡德大学医学中心）；Fraunhofer MEVIS（弗劳恩霍夫梅维斯研究所）；Universität zu Lübeck（吕贝克大学）；University Medical Center Groningen（格罗宁根大学医学中心）；Netherlands Cancer Institute（荷兰癌症研究所）；Norwegian University of Science and Technology（挪威科技大学）
University College London（伦敦大学学院）；UCL Institute of Neurology（伦敦大学学院神经病学研究所）；Wellcome / EPSRC Centre for Interventional and Surgical Sciences, UCL（伦敦大学学院威康/EPSRC介入与外科科学中心）
Image registration between histology and magnetic resonance imaging (MRI) is a challenging task due to differences in structural content and contrast. Too thick and wide specimens cannot be processed all at once and must be cut into smaller pieces. This dramatically increases the complexity of the problem, since each piece should be individually and manually pre-aligned. To the best of our knowledge, no automatic method can reliably locate such piece of tissue within its respective whole in the MRI slice, and align it without any prior information. We propose here a novel automatic approach to…
Large Scale Unsupervised Brain MRI Image Registration Solution for Learn2Reg 2024
College of Electrical and Information Engineering, Hunan University（湖南大学电气与信息工程学院）；National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University（湖南大学机器人视觉感知与控制技术国家工程研究中心）
Multimodal Deformable Image Registration for Long-COVID Analysis Based on Progressive Alignment and Multi-perspective Loss
University of Oxford（牛津大学）；Nuffield Department of Population Health, University of Oxford（牛津大学纳菲尔德人口健康系）；Big Data Institute, University of Oxford（牛津大学大数据研究所）；Oxford University Hospitals NHS Foundation Trust（牛津大学医院 NHS 基金会信托基金）；Oxford Radiology Research Unit（牛津放射学研究单元）
The Wellcome Centre for Human Neuroimaging, UCL（伦敦大学学院惠康人类神经影像中心）
In medical imaging it is common practice to acquire a wide range of modalities (MRI, CT, PET, etc.), to highlight different structures or pathologies. As patient movement between scans or scanning session is unavoidable, registration is often an essential step before any subsequent image analysis. In this paper, we introduce a cost function based on joint total variation for such multimodal image registration. This cost function has the advantage of enabling principled, groupwise alignment of multiple images, whilst being insensitive to strong intensity non-uniformities. We evaluate our algori…
Variational U-Net with Local Alignment for Joint Tumor Extraction and Registration (VALOR-Net) of Breast MRI Data Acquired at Two Different Field Strengths
University of Peshawar（白沙瓦大学）；Danube Private University（多瑙河私立大学）；University of Engineering & Technology, Peshawar（白沙瓦工程与技术大学）；Medical University of Vienna（维也纳医科大学）；Karl Landsteiner Society（卡尔·兰施泰纳学会）；University of Cambridge（剑桥大学）
Rensselaer Polytechnic Institute（伦斯勒理工学院）；National Institutes of Health（美国国立卫生研究院）；National Cancer Institute（美国国家癌症研究所）
Prostate cancer biopsy benefits from accurate fusion of transrectal ultrasound (TRUS) and magnetic resonance (MR) images. In the past few years, convolutional neural networks (CNNs) have been proved powerful in extracting image features crucial for image registration. However, challenging applications and recent advances in computer vision suggest that CNNs are quite limited in its ability to understand spatial correspondence between features, a task in which the self-attention mechanism excels. This paper aims to develop a self-attention mechanism specifically for cross-modal image registrati…
Imperial College London（帝国理工学院）；University of Edinburgh（爱丁堡大学）
Image registration is an ill-posed inverse problem which often requires regularisation on the solution space. In contrast to most of the current approaches which impose explicit regularisation terms such as smoothness, in this paper we propose a novel method that can implicitly learn biomechanics-informed regularisation. Such an approach can incorporate application-specific prior knowledge into deep learning based registration. Particularly, the proposed biomechanics-informed regularisation leverages a variational autoencoder (VAE) to learn a manifold for biomechanically plausible deformations…
A 3D Cross-modal Keypoint Descriptor for MR-US Matching and Registration
Harvard Medical School（哈佛医学院）；Brigham and Women’s Hospital（布莱根妇女医院）；Inria（法国国家信息与自动化研究所）；Sorbonne Université（索邦大学）；Institut du Cerveau - Paris Brain Institute - ICM, CNRS, Inserm, AP-HP, Hôpital de la Pitié Salpêtrière（巴黎大脑研究所 - ICM, 法国国家科学研究中心, 法国国家健康与医学研究院, 巴黎公立医院集团, 皮提耶-萨勒佩特里医院）；Technical University of Munich（慕尼黑工业大学）
Queensland University of Technology（昆士兰科技大学）；CSIRO（联邦科学与工业研究组织）
The recent application of deep learning technologies in medical image registration has exponentially decreased the registration time and gradually increased registration accuracy when compared to their traditional counterparts. Most of the learning-based registration approaches considers this task as a one directional problem. As a result, only correspondence from the moving image to the target image is considered. However, in some medical procedures bidirectional registration is required to be performed. Unlike other learning-based registration, we propose a registration framework with invers…
Zhejiang University（浙江大学）；Shandong Medical Imaging Research Institute（山东医学影像学研究所）；Cheeloo College of Medicine, Shandong University（山东大学齐鲁医学院）；Shanghai Jiao Tong University（上海交通大学）
Multi-slice magnetic resonance images of the fetal brain are usually contaminated by severe and arbitrary fetal and maternal motion. Hence, stable and robust motion correction is necessary to reconstruct high-resolution 3D fetal brain volume for clinical diagnosis and quantitative analysis. However, the conventional registration-based correction has a limited capture range and is insufficient for detecting relatively large motions. Here, we present a novel Affinity Fusion-based Framework for Iteratively Random Motion (AFFIRM) correction of the multi-slice fetal brain MRI. It learns the sequent…
Attention-aware non-rigid image registration for accelerated MR imaging
University Hospital of Tuebingen（蒂宾根大学医院）；Klinikum Rechts der Isar（右岸医院（慕尼黑工业大学附属））；Technical University of Munich（慕尼黑工业大学）；School of Computation, Information and Technology, Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）；Imperial College London（伦敦帝国学院）
Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital（麻省总医院 Athinoula A. Martinos 生物医学影像中心）；Harvard Medical School（哈佛医学院）；Centre for Medical Image Computing, University College London（伦敦大学学院医学影像计算中心）
We introduce a strategy for learning image registration without acquired imaging data, producing powerful networks agnostic to contrast introduced by magnetic resonance imaging (MRI). While classical registration methods accurately estimate the spatial correspondence between images, they solve an optimization problem for every new image pair. Learning-based techniques are fast at test time but limited to registering images with contrasts and geometric content similar to those seen during training. We propose to remove this dependency on training data by leveraging a generative strategy for div…
The Brain Resection Multimodal Image Registration (ReMIND2Reg) 2025 Challenge
Inria Saclay Île-de-France（法国里昂萨克利研究所）；CEA（法国原子能委员会）；Université Paris-Saclay（巴黎-萨克利大学）；Sorbonne Université（索邦大学）；Institut du Cerveau - Paris Brain Institute - ICM（巴黎脑研究所）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化技术研究所）；Inserm（法国国家医学研究院）；AP-HP（法国国家医院集团）；Hôpital de la Pitié Salpêtrière（皮蒂埃-萨尔普里埃尔医院）；Harvard Medical School（哈佛医学院）；Brigham and Women's Hospital（布里根医院）；Department of Radiology and Radiological Science（放射学与放射科学系）；Johns Hopkins Medical School（约翰霍普金斯医学院）；Institute of Medical Informatics（医学信息学研究所）；University of Lübeck（吕贝克大学）；Image Analysis and Communications Laboratory（图像分析与通信实验室）；Department of Electrical and Computer Engineering（电气与计算机工程系）；Johns Hopkins University（约翰霍普金斯大学）；CSAIL（媒体实验室）
University of Bern（伯尔尼大学）；CentraleSupélec（中央理工-高等电力学院）；Université Paris-Saclay（巴黎-萨克雷大学）；Groupe Hospitalier Cochin-Hôtel Dieu（科钦-主宫医院集团）；Université Paris Descartes（巴黎笛卡尔大学）；TheraPanacea
Image registration and in particular deformable registration methods are pillars of medical imaging. Inspired by the recent advances in deep learning, we propose in this paper, a novel convolutional neural network architecture that couples linear and deformable registration within a unified architecture endowed with near real-time performance. Our framework is modular with respect to the global transformation component, as well as with respect to the similarity function while it guarantees smooth displacement fields. We evaluate the performance of our network on the challenging problem of MRI …
Quantib-U（Quantib-U公司）；Amsterdam University Medical Center（阿姆斯特丹大学医学中心）；University of Amsterdam（阿姆斯特丹大学）；University Medical Center Utrecht（乌得勒支大学医学中心）
Deep neural networks are increasingly used for pair-wise image registration. We propose to extend current learning-based image registration to allow simultaneous registration of multiple images. To achieve this, we build upon the pair-wise variational and diffeomorphic VoxelMorph approach and present a general mathematical framework that enables both registration of multiple images to their geodesic average and registration in which any of the available images can be used as a fixed image. In addition, we provide a likelihood based on normalized mutual information, a well-known image similarit…
University of Kansas（堪萨斯大学）；Rochester Institute of Technology（罗切斯特理工学院）
Patient-specific left ventricle (LV) myocardial models have the potential to be used in a variety of clinical scenarios for improved diagnosis and treatment plans. Cine cardiac magnetic resonance (MR) imaging provides high resolution images to reconstruct patient-specific geometric models of the LV myocardium. With the advent of deep learning, accurate segmentation of cardiac chambers from cine cardiac MR images and unsupervised learning for image registration for cardiac motion estimation on a large number of image datasets is attainable. Here, we propose a deep leaning-based framework for th…
Indian Institute of Technology Madras（印度理工学院马德拉斯分校）
In this report we present an unsupervised image registration framework, using a pre-trained deep neural network as a feature extractor. We refer this to zero-shot learning, due to nonoverlap between training and testing dataset (none of the network modules in the processing pipeline were trained specifically for the task of medical image registration). Highlights of our technique are: (a) No requirement of a training dataset (b) Keypoints i.e.locations of important features are automatically estimated (c) The number of key points in this model is fixed and can possibly be tuned as a hyperparam…
DINOMotion: advanced robust tissue motion tracking with DINOv2 in 2D-Cine MRI-guided radiotherapy
Department of Computer Science and Software Engineering, Concordia University（计算机科学与软件工程系，康科迪亚大学）；Department of Electrical and Computer Engineering, Concordia University（电气与计算机工程系，康科迪亚大学）；Elekta Ltd.（Elekta公司）；Diagnos Medical Systems（Diagnos医疗系统）
The Hong Kong University of Science and Technology（香港科技大学）
Recent deep learning-based methods have shown promising results and runtime advantages in deformable image registration. However, analyzing the effects of hyperparameters and searching for optimal regularization parameters prove to be too prohibitive in deep learning-based methods. This is because it involves training a substantial number of separate models with distinct hyperparameter values. In this paper, we propose a conditional image registration method and a new self-supervised learning paradigm for deep deformable image registration. By learning the conditional features that are correla…
Tsinghua University（清华大学）；Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）
Deformable image registration (DIR) is essential for many image-guided therapies. Recently, deep learning approaches have gained substantial popularity and success in DIR. Most deep learning approaches use the so-called mono-stream "high-to-low, low-to-high" network structure, and can achieve satisfactory overall registration results. However, accurate alignments for some severely deformed local regions, which are crucial for pinpointing surgical targets, are often overlooked. Consequently, these approaches are not sensitive to some hard-to-align regions, e.g., intra-patient registration of de…
New York University（纽约大学）；Hyperfine Research（Hyperfine研究公司）；Yale University（耶鲁大学）
Establishing voxelwise semantic correspondence across distinct imaging modalities is a foundational yet formidable computer vision task. Current multi-modality registration techniques maximize hand-crafted inter-domain similarity functions, are limited in modeling nonlinear intensity-relationships and deformations, and may require significant re-engineering or underperform on new tasks, datasets, and domain pairs. This work presents ContraReg, an unsupervised contrastive representation learning approach to multi-modality deformable registration. By projecting learned multi-scale local patch fe…
Recent progress in tissue clearing has allowed for the imaging of entire organs at single-cell resolution. These methods produce very large 3D images (several gigabytes for a whole mouse brain). A necessary step in analysing these images is registration across samples. Existing methods of registration were developed for lower resolution image modalities (e.g. MRI) and it is unclear whether their performance and accuracy is satisfactory at this larger scale. In this study, we used data from different mouse brains cleared with the CUBIC protocol to evaluate five freely available image registrati…
University of Copenhagen（哥本哈根大学）；Biomediq A/S；Lundbeck（灵北）
Template estimation plays a crucial role in computational anatomy since it provides reference frames for performing statistical analysis of the underlying anatomical population variability. While building models for template estimation, variability in sites and image acquisition protocols need to be accounted for. To account for such variability, we propose a generative template estimation model that makes simultaneous inference of both bias fields in individual images, deformations for image registration, and variance hyperparameters. In contrast, existing maximum a posterori based methods ne…
Fast Diffeomorphic Image Registration using Patch based Fully Convolutional Networks
School of Computer and Electrical Engineering, Hunan University of Arts and Science（湖南文理学院计算机与电气工程学院）；Furong College, Hunan University of Arts and Science（湖南文理学院芙蓉学院）；The University of Hong Kong（香港大学）
Image Sciences Institute, University Medical Center Utrecht and Utrecht University（乌得勒支大学医学中心与乌得勒支大学影像科学研究所）；Division of Image Processing of the Leiden University Medical Center（莱顿大学医学中心图像处理部）
Image registration, the process of aligning two or more images, is the core technique of many (semi-)automatic medical image analysis tasks. Recent studies have shown that deep learning methods, notably convolutional neural networks (ConvNets), can be used for image registration. Thus far training of ConvNets for registration was supervised using predefined example registrations. However, obtaining example registrations is not trivial. To circumvent the need for predefined examples, and thereby to increase convenience of training ConvNets for image registration, we propose the Deep Learning Im…
In in-utero MRI, motion correction for fetal body and placenta poses a particular challenge due to the presence of local non-rigid transformations of organs caused by bending and stretching. The existing slice-to-volume registration (SVR) reconstruction methods are widely employed for motion correction of fetal brain that undergoes only rigid transformation. However, for reconstruction of fetal body and placenta, rigid registration cannot resolve the issue of misregistrations due to deformable motion, resulting in degradation of features in the reconstructed volume. We propose a Deformable SVR…
Universidad Politécnica de Madrid（马德里理工大学）；Biomedical Research Networking Center in Bioengineering, Biomaterials and Nanomedicine (CIBER-BBN)（生物医学研究网络中心（生物工程、生物材料与纳米医学））；King’s College London（伦敦国王学院）
Magnetic resonance imaging of whole fetal body and placenta is limited by different sources of motion affecting the womb. Usual scanning techniques employ single-shot multi-slice sequences where anatomical information in different slices may be subject to different deformations, contrast variations or artifacts. Volumetric reconstruction formulations have been proposed to correct for these factors, but they must accommodate a non-homogeneous and non-isotropic sampling, so regularization becomes necessary. Thus, in this paper we propose a deep generative prior for robust volumetric reconstructi…
We present the SIBIA (Scalable Integrated Biophysics-based Image Analysis) framework for joint image registration and biophysical inversion and we apply it to analyse MR images of glioblastomas (primary brain tumors). In particular, we consider the following problem. Given the segmentation of a normal brain MRI and the segmentation of a cancer patient MRI, we wish to determine tumor growth parameters and a registration map so that if we "grow a tumor" (using our tumor model) in the normal segmented image and then register it to the patient segmented image, then the registration mismatch is as …
Computed tomography (CT) is a widely used imaging modality for medical diagnosis and treatment. In electroencephalography (EEG), CT imaging is necessary for co-registering with magnetic resonance imaging (MRI) and for creating more accurate head models for the brain electrical activity due to better representation of bone anatomy. Unfortunately, CT imaging exposes patients to potentially harmful sources of ionizing radiation. Image synthesis methods present a solution for avoiding extra radiation exposure. In this paper, we perform image synthesis to create a realistic, synthetic CT image from…
With a hybrid MEG--MRI device that uses the same sensors for both modalities, the co-registration of MRI and MEG data can be replaced by an automatic calibration step. Based on the highly accurate signal model of ultra-low-field (ULF) MRI, we introduce a calibration method that eliminates the error sources of traditional co-registration. The signal model includes complex sensitivity profiles of the superconducting pickup coils. In ULF MRI, the profiles are independent of the sample and therefore well-defined. In the most basic form, the spatial information of the profiles, captured in parallel…
Wall shear stress (WSS) has been suggested as a potential biomarker in various cardiovascular diseases and it can be estimated from phase-contrast Magnetic Resonance Imaging (PC-MRI) velocity measurements. We present a new parametric sequential method for MRI-based WSS quantification consisting of a geometry identification and a subsequent approximation of the velocity field. The new method was validated on well controlled, high resolution in vitro measurements of turbulent stationary flows and physiological pulsatile flows in phantoms. Basic applicability to in vivo was demonstrated using 2D …
