Image registration is one important task in many image processing applications. It aims to align two or more images so that useful information can be extracted through comparison, combination or superposition. This is achieved by constructing an optimal trans- formation which ensures that the template image becomes similar to a given reference image. Although many models exist, designing a model capable of modelling large and smooth deformation field continues to pose a challenge. This paper proposes a novel variational model for image registration using the Gaussian curvature as a regulariser. The model is motivated by the surface restoration work in geometric processing [Elsey and Esedoglu, Multiscale Model. Simul., (2009), pp. 1549-1573]. An effective numerical solver is provided for the model using an augmented Lagrangian method. Numerical experiments can show that the new model outperforms three competing models based on, respectively, a linear curvature [Fischer and Modersitzki, J. Math. Imaging Vis., (2003), pp. 81- 85], the mean curvature [Chumchob, Chen and Brito, Multiscale Model. Simul., (2011), pp. 89-128] and the diffeomorphic demon model [Vercauteren at al., NeuroImage, (2009), pp. 61-72] in terms of robustness and accuracy.
We present a novel multilevel approach for deep learning based image registration. Recently published deep learning based registration methods have shown promising results for a wide range of tasks. However, these algorithms are still limited to relatively small deformations. Our method addresses this shortcoming by introducing a multilevel framework, which computes deformation fields on different scales, similar to conventional methods. Thereby, a coarse-level alignment is obtained first, which is subsequently improved on finer levels. We demonstrate our method on the complex task of inhale-t…
University of Lübeck（吕贝克大学）；Fraunhofer Institute for Digital Medicine MEVIS（弗劳恩霍夫数字医学研究所 MEVIS）
Image registration, especially the quantification of image similarity, is an important task in image processing. Various approaches for the comparison of two images are discussed in the literature. However, although most of these approaches perform very well in a two image scenario, an extension to a multiple images scenario deserves attention. In this article, we discuss and compare registration methods for multiple images. Our key assumption is, that information about the singular values of a feature matrix of images can be used for alignment. We introduce, discuss and relate three recent ap…
Quantib-U（Quantib-U公司）；Amsterdam University Medical Center（阿姆斯特丹大学医学中心）；University of Amsterdam（阿姆斯特丹大学）；University Medical Center Utrecht（乌得勒支大学医学中心）
Deep neural networks are increasingly used for pair-wise image registration. We propose to extend current learning-based image registration to allow simultaneous registration of multiple images. To achieve this, we build upon the pair-wise variational and diffeomorphic VoxelMorph approach and present a general mathematical framework that enables both registration of multiple images to their geodesic average and registration in which any of the available images can be used as a fixed image. In addition, we provide a likelihood based on normalized mutual information, a well-known image similarit…
Learning a Model-Driven Variational Network for Deformable Image Registration
University of Birmingham（伯明翰大学）；Alan Turing Institute（阿兰·图灵研究所）；Shenzhen University（深圳大学）；Imperial College London（伦敦帝国学院）；MRC London Institute of Medical Sciences（英国医学研究理事会伦敦医学科学研究所）
VPREG: An Optimal Control Formulation for Diffeomorphic Image Registration Based on the Variational Principle Grid Generation Method
Department of Pharmacology and Neuroscience, University of North Texas Health Science Center（药理学与神经科学系，北德克萨斯健康科学中心大学）；Department of Mathematics, University of Texas at Arlington（数学系，德克萨斯大学阿灵顿分校）；Department of Mathematics, University of Houston（数学系，休斯顿大学）
Fraunhofer Institute for Medical Image Computing (MEVIS)（弗劳恩霍夫医学图像计算研究所（MEVIS））；Universität zu Lübeck（吕贝克大学）
We present a highly parallel method for accurate and efficient variational deformable 3D image registration on a consumer-grade graphics processing unit (GPU). We build on recent matrix-free variational approaches and specialize the concepts to the massively-parallel manycore architecture provided by the GPU. Compared to a parallel and optimized CPU implementation, this allows us to achieve an average speedup of 32.53 on 986 real-world CT thorax-abdomen follow-up scans. At a resolution of approximately $256^3$ voxels, the average runtime is 1.99 seconds for the full registration. On the public…
A Bi-variant Variational Model for Diffeomorphic Image Registration with Relaxed Jacobian Determinant Constraints
School of Mathematics and Computational Science（数学与计算科学学院）；University of Strathclyde（思克莱德大学）；Academy of Mathematics and Systems Science（数学与系统科学研究院）；Chinese Academy of Sciences（中国科学院）；National Center for Applied Mathematics in Hunan（湖南省应用数学中心）；Xiangtan University（湘潭大学）
The Institute for Computational Engineering and Sciences, The University of Texas at Austin（德克萨斯大学奥斯汀分校计算工程与科学研究所）
We propose regularization schemes for deformable registration and efficient algorithms for their numerical approximation. We treat image registration as a variational optimal control problem. The deformation map is parametrized by its velocity. Tikhonov regularization ensures well-posedness. Our scheme augments standard smoothness regularization operators based on $H^1$- and $H^2$-seminorms with a constraint on the divergence of the velocity field, which resembles variational formulations for Stokes incompressible flows. In our formulation, we invert for a stationary velocity field and a mass …
Oregon Health and Science University（俄勒冈健康与科学大学）；School of Medicine（医学院）；Department of Science and Engineering（科学与工程系）
We introduce an adaptive regularization approach. In contrast to conventional Tikhonov regularization, which specifies a fixed regularization operator, we estimate it simultaneously with parameters. From a Bayesian perspective we estimate the prior distribution on parameters assuming that it is close to some given model distribution. We constrain the prior distribution to be a Gauss-Markov random field (GMRF), which allows us to solve for the prior distribution analytically and provides a fast optimization algorithm. We apply our approach to non-rigid image registration to estimate the spatial…
Variational U-Net with Local Alignment for Joint Tumor Extraction and Registration (VALOR-Net) of Breast MRI Data Acquired at Two Different Field Strengths
University of Peshawar（白沙瓦大学）；Danube Private University（多瑙河私立大学）；University of Engineering & Technology, Peshawar（白沙瓦工程与技术大学）；Medical University of Vienna（维也纳医科大学）；Karl Landsteiner Society（卡尔·兰施泰纳学会）；University of Cambridge（剑桥大学）
Imperial College London（帝国理工学院）；Technische Universität München（慕尼黑工业大学）；Sharif University of Technology（谢里夫理工大学）；Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡大学）
We develop a new Bayesian model for non-rigid registration of three-dimensional medical images, with a focus on uncertainty quantification. Probabilistic registration of large images with calibrated uncertainty estimates is difficult for both computational and modelling reasons. To address the computational issues, we explore connections between the Markov chain Monte Carlo by backpropagation and the variational inference by backpropagation frameworks, in order to efficiently draw samples from the posterior distribution of transformation parameters. To address the modelling issues, we formulat…
Fraunhofer MEVIS（弗劳恩霍夫医学图像计算研究所）；University of Lübeck（吕贝克大学）
We present a novel computational approach to fast and memory-efficient deformable image registration. In the variational registration model, the computation of the objective function derivatives is the computationally most expensive operation, both in terms of runtime and memory requirements. In order to target this bottleneck, we analyze the matrix structure of gradient and Hessian computations for the case of the normalized gradient fields distance measure and curvature regularization. Based on this analysis, we derive equivalent matrix-free closed-form expressions for derivative computation…
Harvard Medical School（哈佛医学院）；INRIA Sophia Antipolis-Méditerranée（法国国家信息与自动化研究所索菲亚-安提波利斯-地中海中心）；University of North Carolina（北卡罗来纳大学）；Massachusetts Institute of Technology（麻省理工学院）
This paper presents a novel mathematical framework for representing uncertainty in large deformation diffeomorphic image registration. The Bayesian posterior distribution over the deformations aligning a moving and a fixed image is approximated via a variational formulation. A stochastic differential equation (SDE) modeling the deformations as the evolution of a time-varying velocity field leads to a prior density over deformations in the form of a Gaussian process. This permits estimating the full posterior distribution in order to represent uncertainty, in contrast to methods in which the po…
Fraunhofer Institute for Digital Medicine MEVIS（弗劳恩霍夫数字医学研究所MEVIS）；Radboud University Medical Center（拉德堡德大学医学中心）
Deep-learning-based registration methods emerged as a fast alternative to conventional registration methods. However, these methods often still cannot achieve the same performance as conventional registration methods because they are either limited to small deformation or they fail to handle a superposition of large and small deformations without producing implausible deformation fields with foldings inside. In this paper, we identify important strategies of conventional registration methods for lung registration and successfully developed the deep-learning counterpart. We employ a Gaussian-py…
Kuaishou Technology（快手科技）；Cornell University（康奈尔大学）；NVIDIA（英伟达公司）；Tencent（腾讯）；University of California, Irvine（加利福尼亚大学欧文分校）
Registration is a fundamental task in medical robotics and is often a crucial step for many downstream tasks such as motion analysis, intra-operative tracking and image segmentation. Popular registration methods such as ANTs and NiftyReg optimize objective functions for each pair of images from scratch, which are time-consuming for 3D and sequential images with complex deformations. Recently, deep learning-based registration approaches such as VoxelMorph have been emerging and achieve competitive performance. In this work, we construct a test-time training for deep deformable image registratio…
We introduce a region-specific diffeomorphic metric mapping (RDMM) registration approach. RDMM is non-parametric, estimating spatio-temporal velocity fields which parameterize the sought-for spatial transformation. Regularization of these velocity fields is necessary. However, while existing non-parametric registration approaches, e.g., the large displacement diffeomorphic metric mapping (LDDMM) model, use a fixed spatially-invariant regularization our model advects a spatially-varying regularizer with the estimated velocity field, thereby naturally attaching a spatio-temporal regularizer to d…
Fraunhofer MEVIS（弗劳恩霍夫医学图像计算研究所（MEVIS））；University of Lübeck（吕贝克大学）；Anthropics Technology Ltd.（Anthropics科技有限公司）
We present a novel algorithm for the registration of pulmonary CT scans. Our method is designed for large respiratory motion by integrating sparse keypoint correspondences into a dense continuous optimization framework. The detection of keypoint correspondences enables robustness against large deformations by jointly optimizing over a large number of potential discrete displacements, whereas the dense continuous registration achieves subvoxel alignment with smooth transformations. Both steps are driven by the same normalized gradient fields data term. We employ curvature regularization and a v…
University College London（伦敦大学学院）；Verdure Imaging, Inc.（Verdure成像公司）；Queen’s University（女王大学）
Free Point Transformer (FPT) has been proposed as a data-driven, non-rigid point set registration approach using deep neural networks. As FPT does not assume constraints based on point vicinity or correspondence, it may be trained simply and in a flexible manner by minimizing an unsupervised loss based on the Chamfer Distance. This makes FPT amenable to real-world medical imaging applications where ground-truth deformations may be infeasible to obtain, or in scenarios where only a varying degree of completeness in the point sets to be aligned is available. To test the limit of the corresponden…
Bayesian Unsupervised Disentanglement of Anatomy and Geometry for Deep Groupwise Image Registration
School of Data Science, Fudan University（复旦大学数据科学学院）；Department of Electrical and Electronic Engineering and I-X, Imperial College London（帝国理工学院电子与电气工程系）；Department of Electrical and Computer Engineering, University of Washington（华盛顿大学电气与计算机工程系）；Paul G. Allen School of Computer Science and Engineering, University of Washington（华盛顿大学保罗·G·艾伦计算机科学与工程学院）；Department of Radiology and Imaging Sciences, University of Utah（犹他大学放射科与成像科学系）；Department of Radiology, University of Washington（华盛顿大学放射科）；Institute of Science and Technology for Brain-Inspired Intellengence, Fudan University（复旦大学脑启发智能科学技术研究院）
The Institute for Computational Engineering and Sciences, The University of Texas at Austin（得克萨斯大学奥斯汀分校计算工程与科学研究所）；Department of Mathematics at the University of Houston（休斯顿大学数学系）
We propose an efficient numerical algorithm for the solution of diffeomorphic image registration problems. We use a variational formulation constrained by a partial differential equation (PDE), where the constraints are a scalar transport equation. We use a pseudospectral discretization in space and second-order accurate semi-Lagrangian time stepping scheme for the transport equations. We solve for a stationary velocity field using a preconditioned, globalized, matrix-free Newton-Krylov scheme. We propose and test a two-level Hessian preconditioner. We consider two strategies for inverting the…
We propose to learn a probabilistic motion model from a sequence of images for spatio-temporal registration. Our model encodes motion in a low-dimensional probabilistic space - the motion matrix - which enables various motion analysis tasks such as simulation and interpolation of realistic motion patterns allowing for faster data acquisition and data augmentation. More precisely, the motion matrix allows to transport the recovered motion from one subject to another simulating for example a pathological motion in a healthy subject without the need for inter-subject registration. The method is b…
Motion degradation is a central problem in Magnetic Resonance Imaging (MRI). This work addresses the problem of how to obtain higher quality, super-resolved motion-free, reconstructions from highly undersampled MRI data. In this work, we present for the first time a variational multi-task framework that allows joining three relevant tasks in MRI: reconstruction, registration and super-resolution. Our framework takes a set of multiple undersampled MR acquisitions corrupted by motion into a novel multi-task optimisation model, which is composed of an $L^2$ fidelity term that allows sharing repre…
Imperial College London（帝国理工学院）；University of Edinburgh（爱丁堡大学）
Image registration is an ill-posed inverse problem which often requires regularisation on the solution space. In contrast to most of the current approaches which impose explicit regularisation terms such as smoothness, in this paper we propose a novel method that can implicitly learn biomechanics-informed regularisation. Such an approach can incorporate application-specific prior knowledge into deep learning based registration. Particularly, the proposed biomechanics-informed regularisation leverages a variational autoencoder (VAE) to learn a manifold for biomechanically plausible deformations…
In the square root velocity framework, the computation of shape space distances and the registration of curves requires solution of a non-convex variational problem. In this paper, we present a new PDE-based method for solving this problem numerically. The method is constructed from numerical approximation of the Hamilton-Jacobi-Bellman equation for the variational problem, and has quadratic complexity and global convergence for the distance estimate. In conjunction, we propose a backtracking scheme for approximating solutions of the registration problem, which additionally can be used to comp…
In medical image analysis, constructing an atlas, i.e. a mean representative of an ensemble of images, is a critical task for practitioners to estimate variability of shapes inside a population, and to characterise and understand how structural shape changes have an impact on health. This involves identifying significant shape constituents of a set of images, a process called segmentation, and mapping this group of images to an unknown mean image, a task called registration, making a statistical analysis of the image population possible. To achieve this goal, we propose treating these operatio…
A New Variational Model for Joint Image Reconstruction and Motion Estimation in Spatiotemporal Imaging
LSEC, ICMSEC, Academy of Mathematics and Systems Science, Chinese Academy of Sciences（LSEC，ICMSEC，数学系统科学研究所，中国科学院）；Department of Mathematics, KTH–Royal Institute of Technology（数学系，皇家理工学院）
Image registration has played an important role in image processing problems, especially in medical imaging applications. It is well known that when the deformation is large, many variational models cannot ensure diffeomorphism. In this paper, we propose a new registration model based on an optimal control relaxation constraint for large deformation images, which can theoretically guarantee that the registration mapping is diffeomorphic. We present an analysis of optimal control relaxation for indirectly seeking the diffeomorphic transformation of Jacobian determinant equation and its registra…
