Deformable image registration between Computed Tomography (CT) images and Magnetic Resonance (MR) imaging is essential for many image-guided therapies. In this paper, we propose a novel translation-based unsupervised deformable image registration method. Distinct from other translation-based methods that attempt to convert the multimodal problem (e.g., CT-to-MR) into a unimodal problem (e.g., MR-to-MR) via image-to-image translation, our method leverages the deformation fields estimated from both: (i) the translated MR image and (ii) the original CT image in a dual-stream fashion, and automatically learns how to fuse them to achieve better registration performance. The multimodal registration network can be effectively trained by computationally efficient similarity metrics without any ground-truth deformation. Our method has been evaluated on two clinical datasets and demonstrates promising results compared to state-of-the-art traditional and learning-based methods.
Tsinghua University（清华大学）；Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）
Deformable image registration (DIR) is essential for many image-guided therapies. Recently, deep learning approaches have gained substantial popularity and success in DIR. Most deep learning approaches use the so-called mono-stream "high-to-low, low-to-high" network structure, and can achieve satisfactory overall registration results. However, accurate alignments for some severely deformed local regions, which are crucial for pinpointing surgical targets, are often overlooked. Consequently, these approaches are not sensitive to some hard-to-align regions, e.g., intra-patient registration of de…
multiGradICON: A Foundation Model for Multimodal Medical Image Registration
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；University of Salzburg（萨尔茨堡大学）；Université Gustave Eiffel（古斯塔夫·埃菲尔大学）；Brigham and Women’s Hospital（布莱根妇女医院）；ÉTS Montréal（蒙特利尔工程学院）；Boston University（波士顿大学）；Kitware Inc.（Kitware公司）
The Wellcome Centre for Human Neuroimaging, UCL（伦敦大学学院惠康人类神经影像中心）
In medical imaging it is common practice to acquire a wide range of modalities (MRI, CT, PET, etc.), to highlight different structures or pathologies. As patient movement between scans or scanning session is unavoidable, registration is often an essential step before any subsequent image analysis. In this paper, we introduce a cost function based on joint total variation for such multimodal image registration. This cost function has the advantage of enabling principled, groupwise alignment of multiple images, whilst being insensitive to strong intensity non-uniformities. We evaluate our algori…
Dalian University of Technology（大连理工大学）；Guilin University of Electronic Technology（桂林电子科技大学）；Children’s Hospital of Philadelphia（费城儿童医院）；University of Pennsylvania（宾夕法尼亚大学）
Conventional deformable registration methods aim at solving an optimization model carefully designed on image pairs and their computational costs are exceptionally high. In contrast, recent deep learning based approaches can provide fast deformation estimation. These heuristic network architectures are fully data-driven and thus lack explicit geometric constraints, e.g., topology-preserving, which are indispensable to generate plausible deformations. We design a new deep learning based framework to optimize a diffeomorphic model via multi-scale propagation in order to integrate advantages and …
Tsinghua University（清华大学）；Shenzhen International Graduate School（深圳国际研究生院）；Harvard Medical School（哈佛医学院）；Brigham and Women’s Hospital（布莱根妇女医院）
The loss function of an unsupervised multimodal image registration framework has two terms, i.e., a metric for similarity measure and regularization. In the deep learning era, researchers proposed many approaches to automatically learn the similarity metric, which has been shown effective in improving registration performance. However, for the regularization term, most existing multimodal registration approaches still use a hand-crafted formula to impose artificial properties on the estimated deformation field. In this work, we propose a unimodal cyclic regularization training pipeline, which …
Unsupervised Deformable Image Registration with Local-Global Attention and Image Decomposition
Institute of Medical Technology, Peking University Health Science Center（北京大学医学部医学技术研究所）；National Institute of Health Data Science, Peking University（北京大学国家健康数据科学研究院）；Department of Radiology, Peking University Third Hospital（北京大学第三医院放射科）；Department of Pediatrics, Peking University First Hospital（北京大学第一医院儿科）；Pediatric Epilepsy Center, Peking University First Hospital（北京大学第一医院儿童癫痫中心）；School of Biological Science and Medical Engineering, Southeast University（东南大学生物科学与医学工程学院）；Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）
FractMorph: A Fractional Fourier-Based Multi-Domain Transformer for Deformable Image Registration
Faculty of Computer Science and Engineering, Shahid Beheshti University（谢赫·贝赫什特大学计算机科学与工程学院）；School of Computing Science, University of Glasgow（格拉斯哥大学计算机科学学院）；Alan Turing Institute（艾伦·图灵研究所）
Search-MIND: Training-Free Multi-Modal Medical Image Registration
Lab for Uncertainty in Data and Decision Making (LUCID), Nottingham Biomedical Research Centre (BRC), School of Medicine, University of Nottingham, UK（英国诺丁汉大学医学院诺丁汉生物医学研究中心（BRC）数据与决策不确定性实验室（LUCID））
Domain-Shift Immunity in Deep Deformable Registration via Local Feature Representations
Kahlert School of Computing（卡尔伯特计算学校）；Scientific Computing and Imaging Institute（科学计算与成像研究所）；University of Utah（犹他大学）；Department of Biomedical Engineering（生物医学工程系）
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Biomedical Research Imaging Center (BRIC)（生物医学研究成像中心）；UNC Chapel Hill（北卡罗来纳大学教堂山分校）；University of Salzburg（萨尔茨堡大学）
This paper introduces Quicksilver, a fast deformable image registration method. Quicksilver registration for image-pairs works by patch-wise prediction of a deformation model based directly on image appearance. A deep encoder-decoder network is used as the prediction model. While the prediction strategy is general, we focus on predictions for the Large Deformation Diffeomorphic Metric Mapping (LDDMM) model. Specifically, we predict the momentum-parameterization of LDDMM, which facilitates a patch-wise prediction strategy while maintaining the theoretical properties of LDDMM, such as guaranteed…
The Hong Kong University of Science and Technology（香港科技大学）
Diffeomorphic deformable image registration is crucial in many medical image studies, as it offers unique, special properties including topology preservation and invertibility of the transformation. Recent deep learning-based deformable image registration methods achieve fast image registration by leveraging a convolutional neural network (CNN) to learn the spatial transformation from the synthetic ground truth or the similarity metric. However, these approaches often ignore the topology preservation of the transformation and the smoothness of the transformation which is enforced by a global s…
Université de Paris（巴黎大学）；Télécom Paris（巴黎电信学院）；IPParis（巴黎理工学院）
In this paper, we propose an implementation of both Large Deformation Diffeomorphic Metric Mapping (LDDMM) and Metamorphosis image registration using a semi-Lagrangian scheme for geodesic shooting. We propose to solve both problems as an inexact matching providing a single and unifying cost function. We demonstrate that for image registration the use of a semi-Lagrangian scheme is more stable than a standard Eulerian scheme. Our GPU implementation is based on PyTorch, which greatly simplifies and accelerates the computations thanks to its powerful automatic differentiation engine. It will be f…
Current deep-learning-based registration algorithms often exploit intensity-based similarity measures as the loss function, where dense correspondence between a pair of moving and fixed images is optimized through backpropagation during training. However, intensity-based metrics can be misleading when the assumption of intensity class correspondence is violated, especially in cross-modality or contrast-enhanced images. Moreover, existing learning-based registration methods are predominantly applicable to pairwise registration and are rarely extended to groupwise registration or simultaneous re…
McGill University（麦吉尔大学）；Center of Intelligent Machines（智能机器中心）
This paper presents an approach to fast image registration through probabilistic pixel sampling. We propose a practical scheme to leverage the benefits of two state-of-the-art pixel sampling approaches: gradient magnitude based pixel sampling and uniformly random sampling. Our framework involves learning the optimal balance between the two sampling schemes off-line during training, based on a small training dataset, using particle swarm optimization. We then test the proposed sampling approach on 3D rigid registration against two state-of-the-art approaches based on the popular, publicly avail…
Fraunhofer Institute for Medical Image Computing (MEVIS)（弗劳恩霍夫医学图像计算研究所（MEVIS））；Universität zu Lübeck（吕贝克大学）
We present a highly parallel method for accurate and efficient variational deformable 3D image registration on a consumer-grade graphics processing unit (GPU). We build on recent matrix-free variational approaches and specialize the concepts to the massively-parallel manycore architecture provided by the GPU. Compared to a parallel and optimized CPU implementation, this allows us to achieve an average speedup of 32.53 on 986 real-world CT thorax-abdomen follow-up scans. At a resolution of approximately $256^3$ voxels, the average runtime is 1.99 seconds for the full registration. On the public…
University of Bern（伯尔尼大学）；CentraleSupélec（中央理工-高等电力学院）；Université Paris-Saclay（巴黎-萨克雷大学）；Groupe Hospitalier Cochin-Hôtel Dieu（科钦-主宫医院集团）；Université Paris Descartes（巴黎笛卡尔大学）；TheraPanacea
Image registration and in particular deformable registration methods are pillars of medical imaging. Inspired by the recent advances in deep learning, we propose in this paper, a novel convolutional neural network architecture that couples linear and deformable registration within a unified architecture endowed with near real-time performance. Our framework is modular with respect to the global transformation component, as well as with respect to the similarity function while it guarantees smooth displacement fields. We evaluate the performance of our network on the challenging problem of MRI …
Deformable image registration (DIR), aiming to find spatial correspondence between images, is one of the most critical problems in the domain of medical image analysis. In this paper, we present a novel, generic, and accurate diffeomorphic image registration framework that utilizes neural ordinary differential equations (NODEs). We model each voxel as a moving particle and consider the set of all voxels in a 3D image as a high-dimensional dynamical system whose trajectory determines the targeted deformation field. Our method leverages deep neural networks for their expressive power in modeling…
Evaluation of Deformable Image Registration under Alignment-Regularity Trade-off
Technical University of Munich (TUM)（慕尼黑技术大学）；TUM University Hospital, Munich, Germany（慕尼黑技术大学医院）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Department of Computing, Imperial College London（伦敦帝国理工学院计算机系）
University of Texas at Arlington（德克萨斯大学阿灵顿分校）；IBM T. J. Watson Research Center（IBM托马斯·J·沃森研究中心）
In this paper, we propose a novel method for image fusion with a high-resolution panchromatic image and a low-resolution multispectral image at the same geographical location. The fusion is formulated as a convex optimization problem which minimizes a linear combination of a least-squares fitting term and a dynamic gradient sparsity regularizer. The former is to preserve accurate spectral information of the multispectral image, while the latter is to keep sharp edges of the high-resolution panchromatic image. We further propose to simultaneously register the two images during the fusing proces…
Centre de Recherche en Automatique de Nancy（南锡自动化研究中心）；Nancy-University（南锡大学）；CNRS（法国国家科学研究中心）
Bladder cancer is widely spread in the world. Many adequate diagnosis techniques exist. Video-endoscopy remains the standard clinical procedure for visual exploration of the bladder internal surface. However, video-endoscopy presents the limit that the imaged area for each image is about nearly 1cm2. And, lesions are, typically, spread over several images. The aim of this contribution is to assess the performance of two mosaicing algorithms leading to the construction of panoramic maps (one unique image) of bladder walls. The quantitative comparison study is performed on a set of real endoscop…
Indian Institute of Technology Delhi（德里印度理工学院）；Chennai Mathematical Institute（钦奈数学研究所）；Medanta Hospital（梅丹塔医院）
Image registration is a process of aligning two or more images of same objects using geometric transformation. Most of the existing approaches work on the assumption of location invariance. These approaches require object-centric images to perform matching. Further, in absence of intensity level symmetry between the corresponding points in two images, the learning based registration approaches rely on synthetic deformations, which often fail in real scenarios. To address these issues, a combination of convolutional neural networks (CNNs) to perform the desired registration is developed in this…
Neural Graphics Primitives-based Deformable Image Registration for On-the-fly Motion Extraction
Paul Scherrer Institut（保罗·谢勒研究所）；ETH Zurich（苏黎世联邦理工学院）；University Hospital of Zurich（苏黎世大学医院）；University of Bern（伯尔尼大学）；Inselspital, Bern University Hospital（伯尔尼大学医院（ Inselspital））
OSDM-MReg: Multimodal Image Registration based One Step Diffusion Model
Shanghai Key Laboratory of Intelligent Sensing and Recognition（上海智能感知与识别重点实验室）；Shanghai Jiao Tong University（上海交通大学）；Center of Digital Innovation（数字创新中心）
Image registration is a fundamental medical image analysis task, and a wide variety of approaches have been proposed. However, only a few studies have comprehensively compared medical image registration approaches on a wide range of clinically relevant tasks. This limits the development of registration methods, the adoption of research advances into practice, and a fair benchmark across competing approaches. The Learn2Reg challenge addresses these limitations by providing a multi-task medical image registration data set for comprehensive characterisation of deformable registration algorithms. …
Yale University（耶鲁大学）；Yale School of Medicine（耶鲁大学医学院）；University of Science and Technology of China（中国科学技术大学）；Suzhou Institute for Advanced Research, University of Science and Technology of China（中国科学技术大学苏州高等研究院）；Chinese Academy of Sciences（中国科学院）
Multimodal image registration has many applications in diagnostic medical imaging and image-guided interventions, such as Transcatheter Arterial Chemoembolization (TACE) of liver cancer guided by intraprocedural CBCT and pre-operative MR. The ability to register peri-procedurally acquired diagnostic images into the intraprocedural environment can potentially improve the intra-procedural tumor targeting, which will significantly improve therapeutic outcomes. However, the intra-procedural CBCT often suffers from suboptimal image quality due to lack of signal calibration for Hounsfield unit, limi…
University of Warwick（华威大学）；Fraunhofer Institute for Digital Medicine MEVIS（弗劳恩霍夫数字医学MEVIS研究所）；The Alan Turing Institute（艾伦·图灵研究所）；University Hospitals Coventry & Warwickshire（考文垂与沃里克郡大学医院）
Cross-slide image analysis provides additional information by analysing the expression of different biomarkers as compared to a single slide analysis. These biomarker stained slides are analysed side by side, revealing unknown relations between them. During the slide preparation, a tissue section may be placed at an arbitrary orientation as compared to other sections of the same tissue block. The problem is compounded by the fact that tissue contents are likely to change from one section to the next and there may be unique artefacts on some of the slides. This makes registration of each sectio…
Northwest University（西北大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Parcellation of whole-brain tractography streamlines is an important step for tract-based analysis of brain white matter microstructure. Existing fiber parcellation approaches rely on accurate registration between an atlas and the tractograms of an individual, however, due to large individual differences, accurate registration is hard to guarantee in practice. To resolve this issue, we propose a novel deep learning method, called DeepBundle, for registration-free fiber parcellation. Our method utilizes graph convolution neural networks (GCNNs) to predict the parcellation label of each fiber tr…
Distillation Learning Guided by Image Reconstruction for One-Shot Medical Image Segmentation
School of Artificial Intelligence, Beijing University of Posts and Telecommunications（北京邮电大学人工智能学院）；Peking University Third Hospital（北京大学第三医院）；Beijing Children’s Hospital, Capital Medical University（首都医科大学附属北京儿童医院）；Duke University（杜克大学）
In recent years, due to the wide application of multi-sensor vision systems, multimodal image acquisition technology has continued to develop, and the registration problem based on multimodal images has gradually emerged. Most of the existing multimodal image registration methods are only suitable for two modalities, and cannot uniformly register multiple modal image data. Therefore, this paper proposes a multimodal remote sensing image registration method based on adaptive multi-scale PIIFD(AM-PIIFD). This method extracts KAZE features, which can effectively retain edge feature information wh…
University of Edinburgh（爱丁堡大学）；Heriot-Watt University（赫瑞-瓦特大学）
Autofluorescence lifetime images reveal unique characteristics of endogenous fluorescence in biological samples. Comprehensive understanding and clinical diagnosis rely on co-registration with the gold standard, histology images, which is extremely challenging due to the difference of both images. Here, we show an unsupervised image-to-image translation network that significantly improves the success of the co-registration using a conventional optimisation-based regression network, applicable to autofluorescence lifetime images at different emission wavelengths. A preliminary blind comparison …
Images from Positron Emission Tomography (PET) deliver functional data such as perfusion and metabolism. On the other hand, images from Magnetic Resonance Imaging (MRI) provide information describing anatomical structures. Fusing the complementary information from the two modalities is helpful in oncology. In this project, we implemented a complete tool allowing semi-automatic MRI-PET registration for small animal imaging in the preclinical studies. A two-stage hierarchical registration approach is proposed. First, a global affine registration is applied. For robust and fast registration, prin…
CNRS, University of Bordeaux, Bordeaux INP（法国国家科学研究中心、波尔多大学、波尔多综合理工学院）
Affine registration of one or several brain image(s) onto a common reference space is a necessary prerequisite for many image processing tasks, such as brain segmentation or functional analysis. Manual assessment of registration quality is a tedious and time-consuming task, especially in studies comprising a large amount of data. An automated and reliable quality control (QC) becomes mandatory. Moreover, the computation time of the QC must be also compatible with the processing of massive datasets. Therefore, an automated deep neural network approaches appear as a method of choice to automatic…
High-fidelity biomechanical models usually involve the mechanical characterization of biological tissues using experimental methods based on optical measurements. In most experiments, strains are evaluated based on displacements of a few markers and represents an average within the region of interest (ROI). Full-field measurements may improve description of non-homogeneous materials such as soft tissues. The approach based on non-rigid Image Registration is proposed and compared with standard Digital Image Correlation (DIC) on a set of samples, including (i) complex heterogeneous deformations …
