Image Registration and Predictive Modeling: Learning the Metric on the Space of Diffeomorphisms
The Institute for Information Transmission Problems（信息传输问题研究所）；Higher School of Economics（高等经济学院）；University of Pennsylvania（宾夕法尼亚大学）；Indiana University（印第安纳大学）；Northwestern University Feinberg School of Medicine（西北大学范伯格医学院）；Illinois Institute of Technology（伊利诺伊理工学院）
Ayagoz Mussabayeva, Alexey Kroshnin, Anvar Kurmukov, Yulia Dodonova, Li Shen, Shan Cong, Lei Wang, Boris A. Gutman
We present a method for metric optimization in the Large Deformation Diffeomorphic Metric Mapping (LDDMM) framework, by treating the induced Riemannian metric on the space of diffeomorphisms as a kernel in a machine learning context. For simplicity, we choose the kernel Fischer Linear Discriminant Analysis (KLDA) as the framework. Optimizing the kernel parameters in an Expectation-Maximization framework, we define model fidelity via the hinge loss of the decision function. The resulting algorithm optimizes the parameters of the LDDMM norm-inducing differential operator as a solution to a group-wise registration and classification problem. In practice, this may lead to a biology-aware registration, focusing its attention on the predictive task at hand such as identifying the effects of disease. We first tested our algorithm on a synthetic dataset, showing that our parameter selection improves registration quality and classification accuracy. We then tested the algorithm on 3D subcortical shapes from the Schizophrenia cohort Schizconnect. Our Schizpohrenia-Control predictive model showed significant improvement in ROC AUC compared to baseline parameters.
LEDA: Log-Euclidean Diffeomorphism Autoencoder for Efficient Statistical Analysis of Diffeomorphisms
Scientific Computing and Imaging Institute, University of Utah（科学计算与成像研究所，犹他大学）；Kahlert School of Computing, University of Utah（Kahlert计算学院，犹他大学）；Biomedical Engineering Department, University of Utah（生物医学工程系，犹他大学）
Indirect Image Registration with Large Diffeomorphic Deformations
Department of Mathematics, KTH–Royal Institute of Technology（数学系，皇家理工学院）；LSEC, ICMSEC, Academy of Mathematics and Systems Science, Chinese Academy of Sciences（数学和系统科学研究所，中国科学院）
UNC Chapel Hill（北卡罗来纳大学教堂山分校）；University of Salzburg（萨尔茨堡大学）
We present a method to predict image deformations based on patch-wise image appearance. Specifically, we design a patch-based deep encoder-decoder network which learns the pixel/voxel-wise mapping between image appearance and registration parameters. Our approach can predict general deformation parameterizations, however, we focus on the large deformation diffeomorphic metric mapping (LDDMM) registration model. By predicting the LDDMM momentum-parameterization we retain the desirable theoretical properties of LDDMM, while reducing computation time by orders of magnitude: combined with patch pr…
UNC Chapel Hill（北卡罗来纳大学教堂山分校）；University of Salzburg（萨尔茨堡大学）
We introduce a deep encoder-decoder architecture for image deformation prediction from multimodal images. Specifically, we design an image-patch-based deep network that jointly (i) learns an image similarity measure and (ii) the relationship between image patches and deformation parameters. While our method can be applied to general image registration formulations, we focus on the Large Deformation Diffeomorphic Metric Mapping (LDDMM) registration model. By predicting the initial momentum of the shooting formulation of LDDMM, we preserve its mathematical properties and drastically reduce the c…
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Biomedical Research Imaging Center (BRIC)（生物医学研究成像中心）；UNC Chapel Hill（北卡罗来纳大学教堂山分校）；University of Salzburg（萨尔茨堡大学）
This paper introduces Quicksilver, a fast deformable image registration method. Quicksilver registration for image-pairs works by patch-wise prediction of a deformation model based directly on image appearance. A deep encoder-decoder network is used as the prediction model. While the prediction strategy is general, we focus on predictions for the Large Deformation Diffeomorphic Metric Mapping (LDDMM) model. Specifically, we predict the momentum-parameterization of LDDMM, which facilitates a patch-wise prediction strategy while maintaining the theoretical properties of LDDMM, such as guaranteed…
This paper presents a predictive model for estimating regularization parameters of diffeomorphic image registration. We introduce a novel framework that automatically determines the parameters controlling the smoothness of diffeomorphic transformations. Our method significantly reduces the effort of parameter tuning, which is time and labor-consuming. To achieve the goal, we develop a predictive model based on deep convolutional neural networks (CNN) that learns the mapping between pairwise images and the regularization parameter of image registration. In contrast to previous methods that esti…
Institute of Mathematics（数学研究所）；Image Computing, University of Luebeck, Luebeck, Germany（图像计算，吕贝克大学，吕贝克，德国）；Department of Applied Mathematics（应用数学系）；Theoretical Physics, University of Cambridge, Cambridge, United Kingdom（理论物理，剑桥大学，剑桥，英国）
This study proposes an end-to-end unsupervised diffeomorphic deformable registration framework based on moving mesh parameterization. Using this parameterization, a deformation field can be modeled with its transformation Jacobian determinant and curl of end velocity field. The new model of the deformation field has three important advantages; firstly, it relaxes the need for an explicit regularization term and the corresponding weight in the cost function. The smoothness is implicitly embedded in the solution which results in a physically plausible deformation field. Secondly, it guarantees d…
NimbleReg: A light-weight deep-learning framework for diffeomorphic image registration
Hawkes Institute \& Department of Computer Science, University College London, London, UK AINOSTICS ltd., Manchester, UK ESIR, Universit\'e de Rennes, Rennes, France
Image registration is a fundamental step in medical image analysis. Ideally, the transformation that registers one image to another should be a diffeomorphism that is both invertible and smooth. Traditional methods like geodesic shooting approach the problem via differential geometry, with theoretical guarantees that the resulting transformation will be smooth and invertible. Most previous research using unsupervised deep neural networks for registration have used a local smoothness constraint (typically, a spatial variation loss) to address the smoothness issue. These networks usually produce…
VPREG: An Optimal Control Formulation for Diffeomorphic Image Registration Based on the Variational Principle Grid Generation Method
Department of Pharmacology and Neuroscience, University of North Texas Health Science Center（药理学与神经科学系，北德克萨斯健康科学中心大学）；Department of Mathematics, University of Texas at Arlington（数学系，德克萨斯大学阿灵顿分校）；Department of Mathematics, University of Houston（数学系，休斯顿大学）
School of Computer Science, University of Birmingham（伯明翰大学计算机学院）；Institute of Cardiovascular Sciences, University of Birmingham（伯明翰大学心血管科学研究所）；MRC London Institute of Medical Sciences, Imperial College London（帝国理工学院伦敦医学研究委员会医学科学研究所）；Institute of Cancer and Genomic Sciences, University of Birmingham（伯明翰大学癌症与基因组科学研究所）
Deterministic approaches using iterative optimisation have been historically successful in diffeomorphic image registration (DiffIR). Although these approaches are highly accurate, they typically carry a significant computational burden. Recent developments in stochastic approaches based on deep learning have achieved sub-second runtimes for DiffIR with competitive registration accuracy, offering a fast alternative to conventional iterative methods. In this paper, we attempt to reduce this difference in speed whilst retaining the performance advantage of iterative approaches in DiffIR. We firs…
We present an implementation of a new approach to diffeomorphic non-rigid registration of medical images. The method is based on optical flow and warps images via gradient flow with the standard $L^2$ inner product. To compute the transformation, we rely on accelerated optimisation on the manifold of diffeomorphisms. We achieve regularity properties of Sobolev gradient flows, which are expensive to compute, owing to a novel method of averaging the gradients in time rather than space. We successfully register brain MRI and challenging abdominal CT scans at speeds orders of magnitude faster than…
International Institute of Information Technology, Hyderabad（海得拉巴国际信息技术学院）；TIFR Centre for Applicable Mathematics（塔塔基础研究院应用数学中心）
Anatomical variabilities seen in longitudinal data or inter-subject data is usually described by the underlying deformation, captured by non-rigid registration of these images. Stationary Velocity Field (SVF) based non-rigid registration algorithms are widely used for registration. SVF based methods form a metric-free framework which captures a finite dimensional submanifold of deformations embedded in the infinite dimensional smooth manifold of diffeomorphisms. However, these methods cover only a limited degree of deformations. In this paper, we address this limitation and define an approxima…
Télécom Paris, Institut Polytechnique de Paris（巴黎理工学院巴黎电信学院）；CNRS（法国国家科学研究中心）
In medical imaging, most of the image registration methods implicitly assume a one-to-one correspondence between the source and target images (i.e., diffeomorphism). However, this is not necessarily the case when dealing with pathological medical images (e.g., presence of a tumor, lesion, etc.). To cope with this issue, the Metamorphosis model has been proposed. It modifies both the shape and the appearance of an image to deal with the geometrical and topological differences. However, the high computational time and load have hampered its applications so far. Here, we propose a deep residual l…
Masked Registration and Autoencoding of CT Images for Predictive Tibia Reconstruction
Technical University Berlin（技术大学柏林）；Charité – Universitätsmedizin Berlin（柏林查理医院）；Freie Universität Berlin（自由大学柏林）；Humboldt-Universität zu Berlin（洪堡大学柏林）；Institute of Medical Informatics（医学信息学研究所）；Center for Musculoskeletal Surgery（骨科手术中心）；Julius Wolff Institute（朱利叶斯·沃尔夫研究所）
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；University of Southern California（南加州大学）；University of Pennsylvania（宾夕法尼亚大学）；University of Salzburg（萨尔茨堡大学）
Deformable image registration and regression are important tasks in medical image analysis. However, they are computationally expensive, especially when analyzing large-scale datasets that contain thousands of images. Hence, cluster computing is typically used, making the approaches dependent on such computational infrastructure. Even larger computational resources are required as study sizes increase. This limits the use of deformable image registration and regression for clinical applications and as component algorithms for other image analysis approaches. We therefore propose using a fast p…
Aragon Institute on Engineering Research（阿拉贡工程研究所）；University of Zaragoza（萨拉戈萨大学）
The purpose of this work is to contribute to the state of the art of deep-learning methods for diffeomorphic registration. We propose an adversarial learning LDDMM method for pairs of 3D mono-modal images based on Generative Adversarial Networks. The method is inspired by the recent literature for deformable image registration with adversarial learning. We combine the best performing generative, discriminative, and adversarial ingredients from the state of the art within the LDDMM paradigm. We have successfully implemented two models with the stationary and the EPDiff-constrained non-stationar…
We propose a deformable registration algorithm based on unsupervised learning of a low-dimensional probabilistic parameterization of deformations. We model registration in a probabilistic and generative fashion, by applying a conditional variational autoencoder (CVAE) network. This model enables to also generate normal or pathological deformations of any new image based on the probabilistic latent space. Most recent learning-based registration algorithms use supervised labels or deformation models, that miss important properties such as diffeomorphism and sufficiently regular deformation field…
National University of Singapore（新加坡国立大学）；Singapore Institute for Clinical Sciences, Agency for Science, Technology and Research（新加坡科技研究局临床科学研究所）
In this paper, we propose a novel large deformation diffeomorphic registration algorithm to align high angular resolution diffusion images (HARDI) characterized by orientation distribution functions (ODFs). Our proposed algorithm seeks an optimal diffeomorphism of large deformation between two ODF fields in a spatial volume domain and at the same time, locally reorients an ODF in a manner such that it remains consistent with the surrounding anatomical structure. To this end, we first review the Riemannian manifold of ODFs. We then define the reorientation of an ODF when an affine transformatio…
East China Normal University（华东师范大学）；University of Virginia（弗吉尼亚大学）
This paper presents NeurEPDiff, a novel network to fast predict the geodesics in deformation spaces generated by a well known Euler-Poincaré differential equation (EPDiff). To achieve this, we develop a neural operator that for the first time learns the evolving trajectory of geodesic deformations parameterized in the tangent space of diffeomorphisms(a.k.a velocity fields). In contrast to previous methods that purely fit the training images, our proposed NeurEPDiff learns a nonlinear mapping function between the time-dependent velocity fields. A composition of integral operators and smooth act…
Prediction is an appealing objective for self-supervised learning of behavioral skills, particularly for autonomous robots. However, effectively utilizing predictive models for control, especially with raw image inputs, poses a number of major challenges. How should the predictions be used? What happens when they are inaccurate? In this paper, we tackle these questions by proposing a method for learning robotic skills from raw image observations, using only autonomously collected experience. We show that even an imperfect model can complete complex tasks if it can continuously retry, but this …
Mono-Modalizing Extremely Heterogeneous Multi-Modal Medical Image Registration
Department of Computer Science, Yonsei University（延世大学计算机科学系）；Department of Artificial Intelligence, Yonsei University（延世大学人工智能系）；Yonsei University College of Medicine（延世大学医学院）
JUMP: A joint multimodal registration pipeline for neuroimaging with minimal preprocessing
Universitat Politècnica de Catalunya（加泰罗尼亚理工大学）；Institut de Neurociències Department of Biomedicine, University of Barcelona（巴塞罗那大学生物医学系神经科学研究所）；Martinos Center for Biomedical Imaging, MGH and Harvard Medical School（马蒂诺斯生物医学成像中心，麻省总医院和哈佛医学院）；Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology（麻省理工学院计算机科学与人工智能实验室）；Centre for Medical Image Computing, University College London（伦敦大学学院医学图像计算中心）；Centro de Investigación Biomédica en Red de Bioingeniería, Biomateriales y Nanomedicina, ISCIII（生物工程、生物材料与纳米医学生物医学研究中心（ISCIII））；Nuclear Medicine Department, Hospital Clínic（临床医院核医学科）；Institut d’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS)（奥古斯特·皮·苏涅尔生物医学研究所（IDIBAPS））
University of Virginia（弗吉尼亚大学）；Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）；MIT（麻省理工学院）
This paper presents a novel predictive model, MetaMorph, for metamorphic registration of images with appearance changes (i.e., caused by brain tumors). In contrast to previous learning-based registration methods that have little or no control over appearance-changes, our model introduces a new regularization that can effectively suppress the negative effects of appearance changing areas. In particular, we develop a piecewise regularization on the tangent space of diffeomorphic transformations (also known as initial velocity fields) via learned segmentation maps of abnormal regions. The geometr…
