In this paper, we propose a novel method named GP-Aligner to deal with the problem of non-rigid groupwise point set registration. Compared to previous non-learning approaches, our proposed method gains competitive advantages by leveraging the power of deep neural networks to effectively and efficiently learn to align a large number of highly deformed 3D shapes with superior performance. Unlike most learning-based methods that use an explicit feature encoding network to extract the per-shape features and their correlations, our model leverages a model-free learnable latent descriptor to characterize the group relationship. More specifically, for a given group we first define an optimizable Group Latent Descriptor (GLD) to characterize the gruopwise relationship among a group of point sets. Each GLD is randomly initialized from a Gaussian distribution and then concatenated with the coordinates of each point of the associated point sets in the group. A neural network-based decoder is further constructed to predict the coherent drifts as the desired transformation from input groups of shapes to aligned groups of shapes. During the optimization process, GP-Aligner jointly updates all GLDs and weight parameters of the decoder network towards the minimization of an unsupervised groupwise alignment loss. After optimization, for each group our model coherently drives each point set towards a middle, common position (shape) without specifying one as the target. GP-Aligner does not require large-scale training data for network training and it can directly align groups of point sets in a one-stage optimization process. GP-Aligner shows both accuracy and computational efficiency improvement in comparison with state-of-the-art methods for groupwise point set registration. Moreover, GP-Aligner is shown great efficiency in aligning a large number of groups of real-world 3D shapes.
University of Lübeck（吕贝克大学）；Fraunhofer MEVIS（弗劳恩霍夫医学影像计算研究所）
Low-rank and sparse decompositions and robust PCA (RPCA) are highly successful techniques in image processing and have recently found use in groupwise image registration. In this paper, we investigate the drawbacks of the most common RPCA-dissimi\-larity metric in image registration and derive an improved version. In particular, this new metric models low-rank requirements through explicit constraints instead of penalties and thus avoids the pitfalls of the established metric. Equipped with total variation regularization, we present a theoretically justified multilevel scheme based on first-or…
This paper presents a generic probabilistic framework for estimating the statistical dependency and finding the anatomical correspondences among an arbitrary number of medical images. The method builds on a novel formulation of the $N$-dimensional joint intensity distribution by representing the common anatomy as latent variables and estimating the appearance model with nonparametric estimators. Through connection to maximum likelihood and the expectation-maximization algorithm, an information\hyp{}theoretic metric called $\mathcal{X}$-metric and a co-registration algorithm named $\mathcal{X}$…
Fudan University（复旦大学）；University of Washington（华盛顿大学）
Multimodal groupwise registration aligns internal structures in a group of medical images. Current approaches to this problem involve developing similarity measures over the joint intensity profile of all images, which may be computationally prohibitive for large image groups and unstable under various conditions. To tackle these issues, we propose BInGo, a general unsupervised hierarchical Bayesian framework based on deep learning, to learn intrinsic structural representations to measure the similarity of multimodal images. Particularly, a variational auto-encoder with a novel posterior is pr…
Bayesian Unsupervised Disentanglement of Anatomy and Geometry for Deep Groupwise Image Registration
School of Data Science, Fudan University（复旦大学数据科学学院）；Department of Electrical and Electronic Engineering and I-X, Imperial College London（帝国理工学院电子与电气工程系）；Department of Electrical and Computer Engineering, University of Washington（华盛顿大学电气与计算机工程系）；Paul G. Allen School of Computer Science and Engineering, University of Washington（华盛顿大学保罗·G·艾伦计算机科学与工程学院）；Department of Radiology and Imaging Sciences, University of Utah（犹他大学放射科与成像科学系）；Department of Radiology, University of Washington（华盛顿大学放射科）；Institute of Science and Technology for Brain-Inspired Intellengence, Fudan University（复旦大学脑启发智能科学技术研究院）
In this paper, we address the registration of historical WWII images to present-day ortho-photo maps for the purpose of geolocalization. Due to the challenging nature of this problem, we propose to register the images jointly as a group rather than in a step-by-step manner. To this end, we exploit Hough Voting spaces as pairwise registration estimators and show how they can be integrated into a probabilistic groupwise registration framework that can be efficiently optimized. The feature-based nature of our registration framework allows to register images with a-priori unknown translational and…
This paper addresses the task of time separated aerial image registration. The ability to solve this problem accurately and reliably is important for a variety of subsequent image understanding applications. The principal challenge lies in the extent and nature of transient appearance variation that a land area can undergo, such as that caused by the change in illumination conditions, seasonal variations, or the occlusion by non-persistent objects (people, cars). Our work introduces several novelties: (i) unlike all previous work on aerial image registration, we approach the problem using a se…
The Wellcome Centre for Human Neuroimaging, UCL（伦敦大学学院惠康人类神经影像中心）
In medical imaging it is common practice to acquire a wide range of modalities (MRI, CT, PET, etc.), to highlight different structures or pathologies. As patient movement between scans or scanning session is unavoidable, registration is often an essential step before any subsequent image analysis. In this paper, we introduce a cost function based on joint total variation for such multimodal image registration. This cost function has the advantage of enabling principled, groupwise alignment of multiple images, whilst being insensitive to strong intensity non-uniformities. We evaluate our algori…
Université de Lyon（里昂大学）；INSA Lyon（里昂国立应用科学学院）；Université Claude Bernard Lyon 1（里昂第一大学）；UJM-Saint Etienne（圣艾蒂安让·莫内大学）；CNRS（法国国家科学研究中心）；Inserm（法国国家健康与医学研究院）；Hospices Civils de Lyon（里昂民用医院集团）
We present a novel algorithm for Fast Registration Of image Groups (FROG), applied to large 3D image groups. Our approach extracts 3D SURF keypoints from images, computes matched pairs of keypoints and registers the group by minimizing pair distances in a hubless way i.e. without computing any central mean image. Using keypoints significantly reduces the problem complexity compared to voxel-based approaches, and enables us to provide an in-core global optimization, similar to the Bundle Adjustment for 3D reconstruction. As we aim to register images of different patients, the matching step yiel…
Improve Myocardial Strain Estimation based on Deformable Groupwise Registration with a Locally Low-Rank Dissimilarity Metric
Shanghai Jiao Tong University（上海交通大学）；School of Biomedical Engineering（生物医学工程学院）；National Engineering Research Center of Advanced Magnetic Resonance Technologies for Diagnosis and Therapy（国家医学诊疗先进磁共振技术工程研究中心）；Renji Hospital, School of Medicine, Shanghai Jiao Tong University（上海交通大学医学院附属仁济医院）
Center for Image Sciences, University Medical Center Utrecht, the Netherlands（荷兰乌得勒支大学医学中心影像科学中心）；Department of Radiology, University Medical Center Utrecht, the Netherlands（荷兰乌得勒支大学医学中心放射科）
Motion correction of dynamic contrast enhanced magnetic resonance images (DCE-MRI) is a challenging task, due to changes in image appearance. In this study a groupwise registration, using a principle component analysis (PCA) based metric,1 is evaluated for clinical DCE MRI of the liver. The groupwise registration transforms the images to a common space, rather than to a reference volume as conventional pairwise methods do, and computes the similarity metric on all volumes simultaneously. This groupwise registration method is compared to a pairwise approach using a mutual information metric. Cl…
Current deep-learning-based registration algorithms often exploit intensity-based similarity measures as the loss function, where dense correspondence between a pair of moving and fixed images is optimized through backpropagation during training. However, intensity-based metrics can be misleading when the assumption of intensity class correspondence is violated, especially in cross-modality or contrast-enhanced images. Moreover, existing learning-based registration methods are predominantly applicable to pairwise registration and are rarely extended to groupwise registration or simultaneous re…
Biomedical Imaging Research Institute, Cedars-Sinai Medical Center, United States Berlin Ultrahigh Field Facility (B.U.F.F.), Max Delbrück Center for Molecular Medicine in the Helmholtz Association (MDC), Germany Department of Imaging Physics, Delft University of Technology, Netherlands TUM School of Computation, Information；Technology, Technische Universität München, Germany Department of Surgery, Taipei Veterans General Hospital, Taiwan
Multiparametric mapping MRI has become a viable tool for myocardial tissue characterization. However, misalignment between multiparametric maps makes pixel-wise analysis challenging. To address this challenge, we developed a generalizable physics-informed deep-learning model using test-time adaptation to enable group image registration across contrast weighted images acquired from multiple physical models (e.g., a T1 mapping model and T2 mapping model). The physics-informed adaptation utilized the synthetic images from specific physics model as registration reference, allows for transductive l…
Imperial College London（帝国理工学院）；University of Manchester（曼彻斯特大学）
We present a generic method for assessing the quality of non-rigid registration (NRR) algorithms, that does not depend on the existence of any ground truth, but depends solely on the data itself. The data is a set of images. The output of any NRR of such a set of images is a dense correspondence across the whole set. Given such a dense correspondence, it is possible to build various generative statistical models of appearance variation across the set. We show that evaluating the quality of the registration can be mapped to the problem of evaluating the quality of the resultant statistical mode…
CMRINet: Joint Groupwise Registration and Segmentation for Cardiac Function Quantification from Cine-MRI
Leiden University Medical Center（莱顿大学医学中心）；University of Sheffield（谢菲尔德大学）；INSIGNEO；Institute for In Silico Medicine（计算医学研究所）；Department of Infection, Immunity and Cardiovascular Disease（感染、免疫与心血管疾病部门）；Sheffield Teaching Hospitals（谢菲尔德教学医院）；Department of Clinical Radiology（临床放射学部门）
The Kennedy Institute of Rheumatology, University of Oxford（牛津大学肯尼迪风湿病研究所）；Big Data Institute, University of Oxford（牛津大学大数据研究所）；Department of Computer Science, University of Oxford（牛津大学计算机科学系）；Department of Surgery and Cancer, Imperial College London（帝国理工学院外科与癌症系）；Nuffield Department of Population Health, University of Oxford（牛津大学纳菲尔德人口健康系）
Deep learning-based group-wise registration for longitudinal MRI analysis in glioma
Erasmus MC（伊拉斯姆斯大学医学中心）；Medical Delta（医疗三角洲）；Delft University of Technology（代尔夫特理工大学）；Amsterdam UMC / VU Medical Center（阿姆斯特丹大学医学中心/自由大学医学中心）；Princess Máxima Center for Pediatric Oncology（马克西玛公主儿童肿瘤中心）；The Jackson Laboratory for Genomic Medicine（杰克逊基因组医学实验室）
