An Investigation of Feature-based Nonrigid Image Registration using Gaussian Process
FAU Erlangen-Nürnberg（埃尔朗根-纽伦堡大学）；Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）；KRH Klinikum Nordstadt（KRH北城医院）；Universität Klinikum Erlangen（埃尔朗根大学医院）
Siming Bayer, Ute Spiske, Jie Luo, Tobias Geimer, William M. Wells, Martin Ostermeier, Rebecca Fahrig, Arya Nabavi, Christoph Bert, Ilker Eyupoglo, Andreas Maier
For a wide range of clinical applications, such as adaptive treatment planning or intraoperative image update, feature-based deformable registration (FDR) approaches are widely employed because of their simplicity and low computational complexity. FDR algorithms estimate a dense displacement field by interpolating a sparse field, which is given by the established correspondence between selected features. In this paper, we consider the deformation field as a Gaussian Process (GP), whereas the selected features are regarded as prior information on the valid deformations. Using GP, we are able to estimate the both dense displacement field and a corresponding uncertainty map at once. Furthermore, we evaluated the performance of different hyperparameter settings for squared exponential kernels with synthetic, phantom and clinical data respectively. The quantitative comparison shows, GP-based interpolation has performance on par with state-of-the-art B-spline interpolation. The greatest clinical benefit of GP-based interpolation is that it gives a reliable estimate of the mathematical uncertainty of the calculated dense displacement map.
AGH University of Science and Technology（AGH科学技术大学）
In this paper, we present a short description of the method proposed to ANHIR challenge organized jointly with the IEEE ISBI 2019 conference. We propose a method consisting of preprocessing, initial alignment, nonrigid registration algorithms and a method to automatically choose the best result. The method turned out to be robust (99.792% robustness) and accurate (0.38% average median rTRE). The main drawback of the proposed method is relatively high computation time. However, this aspect can be easily improved by cleaning the code and proposing a GPU implementation.
Surface registration, the task of aligning several multidimensional point sets, is a necessary task in many scientific fields. In this work, a novel statistical approach is developed to solve the problem of nonrigid registration. While the application of an affine transformation results in rigid registration, using a general nonlinear function to achieve nonrigid registration is necessary when the point sets require deformations that change over space. The use of a local likelihood-based approach using windowed Gaussian processes provides a flexible way to accurately estimate the nonrigid defo…
Shanghai Jiao Tong University（上海交通大学）；University of the West of England（西英格兰大学）；University College London（伦敦大学学院）
Joint saliency map (JSM) [1] was developed to assign high joint saliency values to the corresponding saliency structures (called Joint Saliency Structures, JSSs) but zero or low joint saliency values to the outliers (or mismatches) that are introduced by missing correspondence or local large deformations between the reference and moving images to be registered. JSM guides the local structure matching in nonrigid registration by emphasizing these JSSs' sparse deformation vectors in adaptive kernel regression of hierarchical sparse deformation vectors for iterative dense deformation reconstructi…
Shanghai Jiao Tong University（上海交通大学）；University College London（伦敦大学学院）；Shanghai Renji Hospital, School of Medicine, Shanghai Jiaotong University（上海交通大学医学院附属上海仁济医院）；University of the West of England（西英格兰大学）
For nonrigid image registration, matching the particular structures (or the outliers) that have missing correspondence and/or local large deformations, can be more difficult than matching the common structures with small deformations in the two images. Most existing works depend heavily on the outlier segmentation to remove the outlier effect in the registration. Moreover, these works do not handle simultaneously the missing correspondences and local large deformations. In this paper, we defined the nonrigid image registration as a local adaptive kernel regression which locally reconstruct the…
Image registration algorithms can be generally categorized into two groups: non-rigid and rigid. Recently, many deep learning-based algorithms employ a neural net to characterize non-rigid image registration function. However, do they always perform better? In this study, we compare the state-of-art deep learning-based non-rigid registration approach with rigid registration approach. The data is generated from Kaggle Dog vs Cat Competition \url{https://www.kaggle.com/c/dogs-vs-cats/} and we test the algorithms' performance on rigid transformation including translation, rotation, scaling, shear…
Leiden University Medical Center（莱顿大学医学中心）；Delft University of Technology（代尔夫特理工大学）；University Medical Center Utrecht（乌得勒支大学医学中心）；TomTom（汤姆汤姆公司）
We propose a supervised nonrigid image registration method, trained using artificial displacement vector fields (DVF), for which we propose and compare three network architectures. The artificial DVFs allow training in a fully supervised and voxel-wise dense manner, but without the cost usually associated with the creation of densely labeled data. We propose a scheme to artificially generate DVFs, and for chest CT registration augment these with simulated respiratory motion. The proposed architectures are embedded in a multi-stage approach, to increase the capture range of the proposed network…
Concordia Institute for Information Systems Engineering（康考迪亚信息系统工程学院）；Concordia University（康考迪亚大学）
We propose a nonrigid registration approach for diffusion tensor images using a multicomponent information-theoretic measure. Explicit orientation optimization is enabled by incorporating tensor reorientation, which is necessary for wrapping diffusion tensor images. Experimental results on diffusion tensor images indicate the feasibility of the proposed approach and a much better performance compared to the affine registration method based on mutual information in terms of registration accuracy in the presence of geometric distortion.
We present a method for nonrigid registration of 2-D geometric shapes. Our contribution is twofold. First, we extend the classic chamfer-matching energy to a variational functional. Secondly, we introduce a meshless deformation model that can handle significant high-curvature deformations. We represent 2-D shapes implicitly using distance transforms, and registration error is defined based on the shape contours' mutual distances. In addition, we model global shape deformation as an approximation blended from local deformation fields using partition-of-unity. The global deformation field is reg…
We propose a framework, named Aggregated Wasserstein, for computing a dissimilarity measure or distance between two Hidden Markov Models with state conditional distributions being Gaussian. For such HMMs, the marginal distribution at any time position follows a Gaussian mixture distribution, a fact exploited to softly match, aka register, the states in two HMMs. We refer to such HMMs as Gaussian mixture model-HMM (GMM-HMM). The registration of states is inspired by the intrinsic relationship of optimal transport and the Wasserstein metric between distributions. Specifically, the components of …
We propose numerical algorithms for solving large deformation diffeomorphic image registration problems. We formulate the nonrigid image registration problem as a problem of optimal control. This leads to an infinite-dimensional partial differential equation (PDE) constrained optimization problem. The PDE constraint consists, in its simplest form, of a hyperbolic transport equation for the evolution of the image intensity. The control variable is the velocity field. Tikhonov regularization on the control ensures well-posedness. We consider standard smoothness regularization based on $H^1$- or …
RegWSI: Whole Slide Image Registration using Combined Deep Feature- and Intensity-Based Methods: Winner of the ACROBAT 2023 Challenge
University of Applied Sciences Western Switzerland（瑞士西应用科学大学）；AGH University of Kraków（克拉科夫AGH大学）；University of Padova（帕多瓦大学）；University of Geneva（日内瓦大学）
We propose a framework, named Aggregated Wasserstein, for computing a dissimilarity measure or distance between two Hidden Markov Models with state conditional distributions being Gaussian. For such HMMs, the marginal distribution at any time spot follows a Gaussian mixture distribution, a fact exploited to softly match, aka register, the states in two HMMs. We refer to such HMMs as Gaussian mixture model-HMM (GMM-HMM). The registration of states is inspired by the intrinsic relationship of optimal transport and the Wasserstein metric between distributions. Specifically, the components of the …
University of Kansas（堪萨斯大学）；Rochester Institute of Technology（罗切斯特理工学院）
Patient-specific left ventricle (LV) myocardial models have the potential to be used in a variety of clinical scenarios for improved diagnosis and treatment plans. Cine cardiac magnetic resonance (MR) imaging provides high resolution images to reconstruct patient-specific geometric models of the LV myocardium. With the advent of deep learning, accurate segmentation of cardiac chambers from cine cardiac MR images and unsupervised learning for image registration for cardiac motion estimation on a large number of image datasets is attainable. Here, we propose a deep leaning-based framework for th…
University of Alberta（阿尔伯塔大学）；Hospital Aleman（阿莱曼医院）
Autograd-based software packages have recently renewed interest in image registration using homography and other geometric models by gradient descent and optimization, e.g., AirLab and DRMIME. In this work, we emphasize on using complex matrix exponential (CME) over real matrix exponential to compute transformation matrices. CME is theoretically more suitable and practically provides faster convergence as our experiments show. Further, we demonstrate that the use of an ordinary differential equation (ODE) as an optimizable dynamical system can adapt the transformation matrix more accurately to…
Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）；Graduate School of Frontier Sciences, The University of Tokyo（东京大学前沿科学研究生院）；Department of Automation, Tsinghua University（清华大学自动化系）；Center for Advanced Intelligence Project, RIKEN（理化学研究所先进智能项目中心）
In image-guided neurosurgery, current commercial systems usually provide only rigid registration, partly because it is harder to predict, validate and understand non-rigid registration error. For instance, when surgeons see a discrepancy in aligned image features, they may not be able to distinguish between registration error and actual tissue deformation caused by tumor resection. In this case, the spatial distribution of registration error could help them make more informed decisions, e.g., ignoring the registration where the estimated error is high. However, error estimates are difficult to…
Fraunhofer Institute for Digital Medicine MEVIS（弗劳恩霍夫数字医学研究所MEVIS）；Radboud University Medical Center（拉德堡德大学医学中心）
Deep-learning-based registration methods emerged as a fast alternative to conventional registration methods. However, these methods often still cannot achieve the same performance as conventional registration methods because they are either limited to small deformation or they fail to handle a superposition of large and small deformations without producing implausible deformation fields with foldings inside. In this paper, we identify important strategies of conventional registration methods for lung registration and successfully developed the deep-learning counterpart. We employ a Gaussian-py…
Harvard Medical School（哈佛医学院）；INRIA Sophia Antipolis-Méditerranée（法国国家信息与自动化研究所索菲亚-安提波利斯-地中海中心）；University of North Carolina（北卡罗来纳大学）；Massachusetts Institute of Technology（麻省理工学院）
This paper presents a novel mathematical framework for representing uncertainty in large deformation diffeomorphic image registration. The Bayesian posterior distribution over the deformations aligning a moving and a fixed image is approximated via a variational formulation. A stochastic differential equation (SDE) modeling the deformations as the evolution of a time-varying velocity field leads to a prior density over deformations in the form of a Gaussian process. This permits estimating the full posterior distribution in order to represent uncertainty, in contrast to methods in which the po…
University of Pennsylvania（宾夕法尼亚大学）；Perelman School of Medicine（佩雷尔曼医学院）；Center for Biomedical Image Computing and Analytics (CBICA)（生物医学图像计算与分析中心（CBICA））
We present a diffeomorphic image registration algorithm to learn spatial transformations between pairs of images to be registered using fully convolutional networks (FCNs) under a self-supervised learning setting. The network is trained to estimate diffeomorphic spatial transformations between pairs of images by maximizing an image-wise similarity metric between fixed and warped moving images, similar to conventional image registration algorithms. It is implemented in a multi-resolution image registration framework to optimize and learn spatial transformations at different image resolutions jo…
Computing Topology Preservation of RBF Transformations for Landmark-Based Image Registration
Department of Mathematics G. Peano , University of Torino（数学系G. Peano，托里诺大学）；Innsbruck Medical University（因斯布鲁克医学院）；University of Applied Science, J. Gutenberg（应用科学大学，J. Gutenberg）
National Laboratory of Radar Signal Processing, Xidian University（西安电子科技大学雷达信号处理国家重点实验室）；School of Computer Science, Shaanxi Normal University（陕西师范大学计算机科学学院）
Nonrigid point set registration is widely applied in the tasks of computer vision and pattern recognition. Coherent point drift (CPD) is a classical method for nonrigid point set registration. However, to solve spatial transformation functions, CPD has to compute inversion of a M*M matrix per iteration with time complexity O(M3). By introducing a simple corresponding constraint, we develop a fast implementation of CPD. The most advantage of our method is to avoid matrix-inverse operation. Before the iteration begins, our method requires to take eigenvalue decomposition of a M*M matrix once. Af…
The clustering for functional data with misaligned problems has drawn much attention in the last decade. Most methods do the clustering after those functional data being registered and there has been little research using both functional and scalar variables. In this paper, we propose a simultaneous registration and clustering (SRC) model via two-level models, allowing the use of both types of variables and also allowing simultaneous registration and clustering. For the data collected from subjects in different unknown groups, a Gaussian process functional regression model with time warping is…
University College London（伦敦大学学院）；University of Bath（巴斯大学）；Google（谷歌）
In this paper we present a dense ground truth dataset of nonrigidly deforming real-world scenes. Our dataset contains both long and short video sequences, and enables the quantitatively evaluation for RGB based tracking and registration methods. To construct ground truth for the RGB sequences, we simultaneously capture Near-Infrared (NIR) image sequences where dense markers - visible only in NIR - represent ground truth positions. This allows for comparison with automatically tracked RGB positions and the formation of error metrics. Most previous datasets containing nonrigidly deforming sequen…
We propose to learn a probabilistic motion model from a sequence of images for spatio-temporal registration. Our model encodes motion in a low-dimensional probabilistic space - the motion matrix - which enables various motion analysis tasks such as simulation and interpolation of realistic motion patterns allowing for faster data acquisition and data augmentation. More precisely, the motion matrix allows to transport the recovered motion from one subject to another simulating for example a pathological motion in a healthy subject without the need for inter-subject registration. The method is b…
Image acquisition and segmentation are likely to introduce noise. Further image processing such as image registration and parameterization can introduce additional noise. It is thus imperative to reduce noise measurements and boost signal. In order to increase the signal-to-noise ratio (SNR) and smoothness of data required for the subsequent random field theory based statistical inference, some type of smoothing is necessary. Among many image smoothing methods, Gaussian kernel smoothing has emerged as a de facto smoothing technique among brain imaging researchers due to its simplicity in numer…
Accounting for 26% of all new cancer cases worldwide, breast cancer remains the most common form of cancer in women. Although early breast cancer has a favourable long-term prognosis, roughly a third of patients suffer from a suboptimal aesthetic outcome despite breast conserving cancer treatment. Clinical-quality 3D modelling of the breast surface therefore assumes an increasingly important role in advancing treatment planning, prediction and evaluation of breast cosmesis. Yet, existing 3D torso scanners are expensive and either infrastructure-heavy or subject to motion artefacts. In this pap…
Bayesian analysis of functions and curves is considered, where warping and other geometrical transformations are often required for meaningful comparisons. We focus on two applications involving the classification of mouse vertebrae shape outlines and the alignment of mass spectrometry data in proteomics. The functions and curves of interest are represented using the recently introduced square root velocity function, which enables a warping invariant elastic distance to be calculated in a straightforward manner. We distinguish between various spaces of interest: the original space, the ambient…
Bayesian In Vivo Tracking of Synapses using Joint Poisson Deconvolution and Diffeomorphic Registration
Department of Biomedical Engineering, Johns Hopkins University（约翰霍普金斯大学生物医学工程系）；Department of Neuroscience, Johns Hopkins University（约翰霍普金斯大学神经科学系）；Kavli Neuroscience Discovery Institute, Johns Hopkins University（约翰霍普金斯大学Kavli神经科学发现研究所）；Data Science and AI Institute, Johns Hopkins University（约翰霍普金斯大学数据科学与人工智能研究所）；Department of Applied Mathematics and Statistics, Johns Hopkins University（约翰霍普金斯大学应用数学与统计学系）
Information Systems Institute, University of Applied Sciences Western Switzerland（西瑞士应用科学大学信息系统学院）；AGH University of Kraków（克拉科夫AGH大学）；University of Padova（帕多瓦大学）；University of Geneva（日内瓦大学）
Charles Sturt University（查尔斯特大学）；Study Group Australia（澳大利亚学习集团）；University of Technology, Baghdad（巴格达技术大学）；University of Kurdistan Hewler（库尔德斯坦赫勒大学）
Background and aim: Image registration and alignment are the main limitations of augmented reality-based knee replacement surgery. This research aims to decrease the registration error, eliminate outcomes that are trapped in local minima to improve the alignment problems, handle the occlusion, and maximize the overlapping parts. Methodology: markerless image registration method was used for Augmented reality-based knee replacement surgery to guide and visualize the surgical operation. While weight least square algorithm was used to enhance stereo camera-based tracking by filling border occlusi…
Gaussian noise removal is an interesting area in digital image processing not only to improve the visual quality, but for its impact on other post-processing algorithms like image registration or segmentation. Many presented state-of-the-art denoising methods are based on the self-similarity or patch-based image processing. Specifically, Non-Local Means (NLM) as a patch-based filter has gained increasing attention in recent years. Essentially, this filter tends to obtain the noise-less signal value by computing the Gaussian-weighted Euclidean distance between the patch under-processing and oth…
Covariance operators are fundamental in functional data analysis, providing the canonical means to analyse functional variation via the celebrated Karhunen--Loève expansion. These operators may themselves be subject to variation, for instance in contexts where multiple functional populations are to be compared. Statistical techniques to analyse such variation are intimately linked with the choice of metric on covariance operators, and the intrinsic infinite-dimensionality of these operators. In this paper, we describe the manifold geometry of the space of trace-class infinite-dimensional covar…
University College London（伦敦大学学院）；Harvard Medical School（哈佛医学院）；Massachusetts General Hospital（麻省总医院）；Allen Institute for Brain Science（艾伦脑科学研究所）；Massachusetts Institute of Technology（麻省理工学院）
Nonlinear registration of 2D histological sections with corresponding slices of MRI data is a critical step of 3D histology reconstruction. This task is difficult due to the large differences in image contrast and resolution, as well as the complex nonrigid distortions produced when sectioning the sample and mounting it on the glass slide. It has been shown in brain MRI registration that better spatial alignment across modalities can be obtained by synthesizing one modality from the other and then using intra-modality registration metrics, rather than by using mutual information (MI) as metric…
Unsupervised Method for Intra-patient Registration of Brain Magnetic Resonance Images based on Objective Function Weighting by Inverse Consistency: Contribution to the BraTS-Reg Challenge
University of Applied Sciences Western Switzerland（瑞士西部应用科学大学）；AGH University of Science and Technology（AGH科技大学）；University of Geneva（日内瓦大学）；University of Padova（帕多瓦大学）
Point cloud registration is a fundamental problem in 3D computer vision, graphics and robotics. For the last few decades, existing registration algorithms have struggled in situations with large transformations, noise, and time constraints. In this paper, we introduce Deep Gaussian Mixture Registration (DeepGMR), the first learning-based registration method that explicitly leverages a probabilistic registration paradigm by formulating registration as the minimization of KL-divergence between two probability distributions modeled as mixtures of Gaussians. We design a neural network that extract…
In this paper, we unify popular non-rigid registration methods for point sets and surfaces under our general framework, GiNGR. GiNGR builds upon Gaussian Process Morphable Models (GPMM) and hence separates modeling the deformation prior from model adaptation for registration. In addition, it provides explainable hyperparameters, multi-resolution registration, trivial inclusion of expert annotation, and the ability to use and combine analytical and statistical deformation priors. But more importantly, the reformulation allows for a direct comparison of registration methods. Instead of using a g…
University of Technology Sydney（悉尼科技大学）；Fondazione Bruno Kessler（布鲁诺·凯塞勒基金会）；University of Trento（特伦托大学）
Probabilistic 3D point cloud registration methods have shown competitive performance in overcoming noise, outliers, and density variations. However, registering point cloud pairs in the case of partial overlap is still a challenge. This paper proposes a novel overlap-guided probabilistic registration approach that computes the optimal transformation from matched Gaussian Mixture Model (GMM) parameters. We reformulate the registration problem as the problem of aligning two Gaussian mixtures such that a statistical discrepancy measure between the two corresponding mixtures is minimized. We intro…
We present a flexible approach to colour transfer inspired by techniques recently proposed for shape registration. Colour distributions of the palette and target images are modelled with Gaussian Mixture Models (GMMs) that are robustly registered to infer a non linear parametric transfer function. We show experimentally that our approach compares well to current techniques both quantitatively and qualitatively. Moreover, our technique is computationally the fastest and can take efficient advantage of parallel processing architectures for recolouring images and videos. Our transfer function is …
Probabilistic point-set registration methods have been gaining more attention for their robustness to noise, outliers and occlusions. However, these methods tend to be much slower than the popular iterative closest point (ICP) algorithms, which severely limits their usability. In this paper, we contribute a novel probabilistic registration method that achieves state-of-the-art robustness as well as substantially faster computational performance than modern ICP implementations. This is achieved using a rigorous yet computationally-efficient probabilistic formulation. Point-set registration is c…
The goal of point set registration is to find point-by-point correspondences between point sets, each of which characterizes the shape of an object. Because local preservation of object geometry is assumed, prevalent algorithms in the area can often elegantly solve the problems without using geometric information specific to the objects. This means that registration performance can be further improved by using prior knowledge of object geometry. In this paper, we propose a novel point set registration method using the Gaussian mixture model with prior shape information encoded as a statistical…
Australian National University（澳大利亚国立大学）；National ICT Australia (NICTA)（澳大利亚国家信息通信技术中心）
This paper presents a framework for rigid point-set registration and merging using a robust continuous data representation. Our point-set representation is constructed by training a one-class support vector machine with a Gaussian radial basis function kernel and subsequently approximating the output function with a Gaussian mixture model. We leverage the representation's sparse parametrisation and robustness to noise, outliers and occlusions in an efficient registration algorithm that minimises the L2 distance between our support vector--parametrised Gaussian mixtures. In contrast, existing t…
