Remote sensing image registration is valuable for image-based navigation system despite posing many challenges. As the search space of registration is usually non-convex, the optimization algorithm, which aims to search the best transformation parameters, is a challenging step. Conventional optimization algorithms can hardly reconcile the contradiction of simultaneous rapid convergence and the global optimization. In this paper, a novel learning-based optimization algorithm named Image Registration Optimizer Network (IRON) is proposed, which can predict the global optimum after single iteration. The IRON is trained by a 3D tensor (9x9x9), which consists of similar metric values. The elements of the 3D tensor correspond to the 9x9x9 neighbors of the initial parameters in the search space. Then, the tensor's label is a vector that points to the global optimal parameters from the initial parameters. Because of the special architecture, the IRON could predict the global optimum directly for any initialization. The experimental results demonstrate that the proposed algorithm performs better than other classical optimization algorithms as it has higher accuracy, lower root of mean square error (RMSE), and more efficiency. Our IRON codes are available for further study.https://www.github.com/jaxwangkd04/IRON
Factored Levenberg-Marquardt for Diffeomorphic Image Registration: An efficient optimizer for FireANTs
Department of Computer and Information Science（计算机与信息科学系）；University of Pennsylvania（宾夕法尼亚大学）；Department of Electrical and Systems Engineering（电气与系统工程系）；Department of Radiology & Penn Image Computing and Science Laboratory (PICSL)（放射科及宾夕法尼亚大学图像计算与科学实验室（PICSL））；Perelman School of Medicine, University of Pennsylvania（佩尔曼医学学院，宾夕法尼亚大学）
University of Edinburgh（爱丁堡大学）；Toshiba Medical Visualization Systems（东芝医疗可视化系统公司）
In this paper we describe improvements to the particle swarm optimizer (PSO) made by inclusion of an unscented Kalman filter to guide particle motion. We demonstrate the effectiveness of the unscented Kalman filter PSO by comparing it with the original PSO algorithm and its variants designed to improve performance. The PSOs were tested firstly on a number of common synthetic benchmarking functions, and secondly applied to a practical three-dimensional image registration problem. The proposed methods displayed better performances for 4 out of 8 benchmark functions, and reduced the target regist…
Centre for Medical Image Computing, University College London（伦敦大学学院医学图像计算中心）；Wellcome/EPSRC Centre for Surgical & Interventional Sciences, University College London（伦敦大学学院惠康/工程与物理科学研究委员会外科与介入科学中心）
Neural networks have been proposed for medical image registration by learning, with a substantial amount of training data, the optimal transformations between image pairs. These trained networks can further be optimized on a single pair of test images - known as test-time optimization. This work formulates image registration as a meta-learning algorithm. Such networks can be trained by aligning the training image pairs while simultaneously improving test-time optimization efficacy; tasks which were previously considered two independent training and optimization processes. The proposed meta-reg…
This letter proposes a synthetic aperture radar (SAR) image registration method named Feature-Area Optimization (FAO). First, the traditional area-based optimization model is reconstructed and decomposed into three key but uncertain factors: initialization, slice set and regularization. Next, structural features are extracted by scale invariant feature transform (SIFT) in dual-resolution space (SIFT-DRS), a novel SIFT-Like method dedicated to FAO. Then, the three key factors are determined based on these features. Finally, solving the factor-determined optimization model can get the registrati…
McGill University（麦吉尔大学）；Center of Intelligent Machines（智能机器中心）
This paper presents an approach to fast image registration through probabilistic pixel sampling. We propose a practical scheme to leverage the benefits of two state-of-the-art pixel sampling approaches: gradient magnitude based pixel sampling and uniformly random sampling. Our framework involves learning the optimal balance between the two sampling schemes off-line during training, based on a small training dataset, using particle swarm optimization. We then test the proposed sampling approach on 3D rigid registration against two state-of-the-art approaches based on the popular, publicly avail…
Image registration is the process of transforming different sets of data into one coordinate system and is required for various remote sensing applications like change detection, image fusion, and other related areas. The effect of increased relief displacement, requirement of more control points, and increased data volume are the challenges associated with the registration of high resolution image data. The objective of this research work is to study the most efficient techniques and to investigate the extent of improvement achievable by enhancing them with Wavelet transform. The SIFT feature…
Dalian University of Technology（大连理工大学）；Guilin University of Electronic Technology（桂林电子科技大学）；Children’s Hospital of Philadelphia（费城儿童医院）；University of Pennsylvania（宾夕法尼亚大学）
Conventional deformable registration methods aim at solving an optimization model carefully designed on image pairs and their computational costs are exceptionally high. In contrast, recent deep learning based approaches can provide fast deformation estimation. These heuristic network architectures are fully data-driven and thus lack explicit geometric constraints, e.g., topology-preserving, which are indispensable to generate plausible deformations. We design a new deep learning based framework to optimize a diffeomorphic model via multi-scale propagation in order to integrate advantages and …
SAMConvex: Fast Discrete Optimization for CT Registration using Self-supervised Anatomical Embedding and Correlation Pyramid
DAMO Academy, Alibaba Group（达摩院，阿里巴巴集团）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；The First Affiliated Hospital of College of Medicine, Zhejiang University（浙江大学医学院附属第一医院）；Hupan Lab（华潘实验室）
Automatic Registration of SHG and H&E Images with Feature-based Initial Alignment and Intensity-based Instance Optimization: Contribution to the COMULIS Challenge
AGH University of Krakow（克拉科夫AGH科技大学）；University of Applied Sciences Western Switzerland (HES-SO Valais-Wallis)（瑞士西部应用科学大学（瓦莱州-瓦利斯））；University of Geneva（日内瓦大学）
Unsupervised MR-US Multimodal Image Registration with Multilevel Correlation Pyramidal Optimization
School of Artificial Intelligence and Robotics, Hunan University, Changsha, Hunan, China（人工智能与机器人学院，湖南大学，长沙，湖南，中国）；National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, Changsha, Hunan, China（机器人视觉感知与控制技术国家工程研究中心，湖南大学，长沙，湖南，中国）；National University of Singapore（新加坡国立大学）；Cornell University（康奈尔大学）
University of California, Irvine（加州大学欧文分校）；Yale University（耶鲁大学）
Diffeomorphic image registration, offering smooth transformation and topology preservation, is required in many medical image analysis tasks.Traditional methods impose certain modeling constraints on the space of admissible transformations and use optimization to find the optimal transformation between two images. Specifying the right space of admissible transformations is challenging: the registration quality can be poor if the space is too restrictive, while the optimization can be hard to solve if the space is too general. Recent learning-based methods, utilizing deep neural networks to lea…
Rensselaer Polytechnic Institute（伦斯勒理工学院）；Philips Research North America（飞利浦北美研究院）；National Institutes of Health（美国国立卫生研究院）
Purpose: The fusion of transrectal ultrasound (TRUS) and magnetic resonance (MR) images for guiding targeted prostate biopsy has significantly improved the biopsy yield of aggressive cancers. A key component of MR-TRUS fusion is image registration. However, it is very challenging to obtain a robust automatic MR-TRUS registration due to the large appearance difference between the two imaging modalities. The work presented in this paper aims to tackle this problem by addressing two challenges: (i) the definition of a suitable similarity metric and (ii) the determination of a suitable optimizatio…
Ecole Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）
We present a performance analysis for image registration with gradient descent methods. We consider a typical multiscale registration setting where the global 2-D translation between a pair of images is estimated by smoothing the images and minimizing the distance between them with gradient descent. Our study particularly concentrates on the effect of noise and low-pass filtering on the alignment accuracy. We adopt an analytic representation for images and analyze the well-behavedness of the image distance function by estimating the neighborhood of translations for which it is free of undesire…
Imperial College London（帝国理工学院）；Technical University of Munich（慕尼黑工业大学）；University of Edinburgh（爱丁堡大学）
Deep learning (DL) image registration methods amortize the costly pair-wise iterative optimization by training deep neural networks to predict the optimal transformation in one fast forward-pass. In this work, we bridge the gap between traditional iterative energy optimization-based registration and network-based registration, and propose Gradient Descent Network for Image Registration (GraDIRN). Our proposed approach trains a DL network that embeds unrolled multiresolution gradient-based energy optimization in its forward pass, which explicitly enforces image dissimilarity minimization in its…
Deformable image registration (DIR), aiming to find spatial correspondence between images, is one of the most critical problems in the domain of medical image analysis. In this paper, we present a novel, generic, and accurate diffeomorphic image registration framework that utilizes neural ordinary differential equations (NODEs). We model each voxel as a moving particle and consider the set of all voxels in a 3D image as a high-dimensional dynamical system whose trajectory determines the targeted deformation field. Our method leverages deep neural networks for their expressive power in modeling…
UT Southwestern Medical Center（德克萨斯大学西南医学中心）；Southern Methodist University（南方卫理公会大学）
Medical image registration is a fundamental and vital task which will affect the efficacy of many downstream clinical tasks. Deep learning (DL)-based deformable image registration (DIR) methods have been investigated, showing state-of-the-art performance. A test time optimization (TTO) technique was proposed to further improve the DL models' performance. Despite the substantial accuracy improvement with this TTO technique, there still remained some regions that exhibited large registration errors even after many TTO iterations. To mitigate this challenge, we firstly identified the reason why t…
Primitive Simultaneous Optimization of Similarity Metrics for Image Registration
Technical University of Munich（慕尼黑工业大学）；University of Zurich（苏黎世大学）；Klinikum rechts der Isar（伊萨尔河右岸医院）；TranslaTUM - Central Institute for Translational Cancer Research（TranslaTUM——转化癌症研究中央研究所）
UNC Chapel Hill（北卡罗来纳大学教堂山分校）；University of Salzburg（萨尔茨堡大学）
We introduce a deep encoder-decoder architecture for image deformation prediction from multimodal images. Specifically, we design an image-patch-based deep network that jointly (i) learns an image similarity measure and (ii) the relationship between image patches and deformation parameters. While our method can be applied to general image registration formulations, we focus on the Large Deformation Diffeomorphic Metric Mapping (LDDMM) registration model. By predicting the initial momentum of the shooting formulation of LDDMM, we preserve its mathematical properties and drastically reduce the c…
Robarts Research Institute（罗巴茨研究所）；Western University（韦仕敦大学）
Optimization techniques have been widely used in deformable registration, allowing for the incorporation of similarity metrics with regularization mechanisms. These regularization mechanisms are designed to mitigate the effects of trivial solutions to ill-posed registration problems and to otherwise ensure the resulting deformation fields are well-behaved. This paper introduces a novel deformable registration algorithm, RANCOR, which uses iterative convexification to address deformable registration problems under total-variation regularization. Initial comparative results against four state-of…
Deep Learning in Medical Image Registration: Magic or Mirage?
Computer and Information Science（计算机与信息科学学院）；Electrical and Systems Engineering（电气与系统工程学院）；Radiology（放射学系）；Penn Image Computing and Science Laboratory（宾夕法尼亚图像计算与科学实验室）
University of Copenhagen（哥本哈根大学）；Technical University of Denmark（丹麦技术大学）
We propose a semantic similarity metric for image registration. Existing metrics like euclidean distance or normalized cross-correlation focus on aligning intensity values, giving difficulties with low intensity contrast or noise. Our semantic approach learns dataset-specific features that drive the optimization of a learning-based registration model. Comparing to existing unsupervised and supervised methods across multiple image modalities and applications, we achieve consistently high registration accuracy and faster convergence than state of the art, and the learned invariance to noise give…
Image registration plays an important role in medical image analysis. Conventional optimization based methods provide an accurate estimation due to the iterative process at the cost of expensive computation. Deep learning methods such as learn-to-map are much faster but either iterative or coarse-to-fine approach is required to improve accuracy for handling large motions. In this work, we proposed to learn a registration optimizer via a multi-scale neural ODE model. The inference consists of iterative gradient updates similar to a conventional gradient descent optimizer but in a much faster wa…
Objective: Deformable image registration is a fundamental problem in medical image analysis, with applications such as longitudinal studies, population modeling, and atlas based image segmentation. Registration is often phrased as an optimization problem, i.e., finding a deformation field that is optimal according to a given objective function. Discrete, combinatorial, optimization techniques have successfully been employed to solve the resulting optimization problem. Specifically, optimization based on $α$-expansion with minimal graph cuts has been proposed as a powerful tool for image regist…
We present Deep Global Registration, a differentiable framework for pairwise registration of real-world 3D scans. Deep global registration is based on three modules: a 6-dimensional convolutional network for correspondence confidence prediction, a differentiable Weighted Procrustes algorithm for closed-form pose estimation, and a robust gradient-based SE(3) optimizer for pose refinement. Experiments demonstrate that our approach outperforms state-of-the-art methods, both learning-based and classical, on real-world data.
The Discriminative Optimization (DO) algorithm has been proved much successful in 3D point cloud registration. In the original DO, the feature (descriptor) of two point cloud was defined as a histogram, and the element of histogram indicates the weights of scene points in "front" or "back" side of a model point. In this paper, we extended the histogram which indicate the sides from "front-back" to "front-back", "up-down", and "clockwise-anticlockwise". In addition, we reweighted the extended histogram according to the model points' distribution. We evaluated the proposed Improved DO on the Sta…
On-the-Fly Guidance Training for Medical Image Registration
University of Leeds（利兹大学）；University of California, Irvine（加州大学欧文分校）；Tongji University（同济大学）；Huazhong University of Science and Technology（华中科技大学）；Fudan Univeristy（复旦大学）；University of California, San Diego（加州大学圣迭戈分校）
The University of Sydney（悉尼大学）；University of Technology Sydney（悉尼科技大学）
Registration is a transformation estimation problem between two point clouds, which has a unique and critical role in numerous computer vision applications. The developments of optimization-based methods and deep learning methods have improved registration robustness and efficiency. Recently, the combinations of optimization-based and deep learning methods have further improved performance. However, the connections between optimization-based and deep learning methods are still unclear. Moreover, with the recent development of 3D sensors and 3D reconstruction techniques, a new research directio…
Technion – Israel Institute of Technology（以色列理工学院）
Quantification of uncertainty in deep-neural-networks (DNN) based image registration algorithms plays a critical role in the deployment of image registration algorithms for clinical applications such as surgical planning, intraoperative guidance, and longitudinal monitoring of disease progression or treatment efficacy as well as in research-oriented processing pipelines. Currently available approaches for uncertainty estimation in DNN-based image registration algorithms may result in sub-optimal clinical decision making due to potentially inaccurate estimation of the uncertainty of the registr…
Unsupervised Multimodal 3D Medical Image Registration with Multilevel Correlation Balanced Optimization
College of Electrical and Information Engineering, Hunan University（湖南大学电气与信息工程学院）；National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University（湖南大学机器人视觉感知与控制技术国家工程研究中心）；Cornell University（康奈尔大学）
Tsinghua-Berkeley Shenzhen Institute（清华-伯克利深圳学院）；Hong Kong University of Science and Technology（香港科技大学）；Technical University of Munich（慕尼黑工业大学）
Direct image-to-image alignment that relies on the optimization of photometric error metrics suffers from limited convergence range and sensitivity to lighting conditions. Deep learning approaches has been applied to address this problem by learning better feature representations using convolutional neural networks, yet still require a good initialization. In this paper, we demonstrate that the inaccurate numerical Jacobian limits the convergence range which could be improved greatly using learned approaches. Based on this observation, we propose a novel end-to-end network, RegNet, to learn th…
Indian Institute of Information Technology, Sri City（斯里城印度信息技术学院）；Indian Institute of Information Technology, Allahabad（阿拉哈巴德印度信息技术学院）
Convolutional neural networks (CNNs) have shown very appealing performance for many computer vision applications. The training of CNNs is generally performed using stochastic gradient descent (SGD) based optimization techniques. The adaptive momentum-based SGD optimizers are the recent trends. However, the existing optimizers are not able to maintain a zero mean in the first-order moment and struggle with optimization. In this paper, we propose a moment centralization-based SGD optimizer for CNNs. Specifically, we impose the zero mean constraints on the first-order moment explicitly. The propo…
Faculty of Electrical Engineering and Computing（电气工程与计算学院）
Surface registration is a technique that is used in various areas such as object recognition and 3D model reconstruction. Problem of surface registration can be analyzed as an optimization problem of seeking a rigid motion between two different views. Genetic algorithms can be used for solving this optimization problem, both for obtaining the robust parameter estimation and for its fine-tuning. The main drawback of genetic algorithms is that they are time consuming which makes them unsuitable for online applications. Modern acquisition systems enable the implementation of the solutions that wo…
Accurate Reconstruction of Gas Turbine Blade Geometry Using 3D/2D Rigid Registration and CT View Optimization
École Polytechnique de Montréal（蒙特利尔综合理工学院）；Object Research Systems Inc.（对象研究系统公司）；École de technologie supérieure（高等技术学院）；Pratt & Whitney Canada（加拿大普惠公司）
Institute of Medical Informatics, University of Lübeck（吕贝克大学医学信息学研究所）
Recent deep learning-based methods for medical image registration achieve results that are competitive with conventional optimization algorithms at reduced run times. However, deep neural networks generally require plenty of labeled training data and are vulnerable to domain shifts between training and test data. While typical intensity shifts can be mitigated by keypoint-based registration, these methods still suffer from geometric domain shifts, for instance, due to different fields of view. As a remedy, in this work, we present a novel approach to geometric domain adaptation for image regis…
Concordia Institute for Information Systems Engineering（康考迪亚信息系统工程学院）；Concordia University（康考迪亚大学）
We propose a nonrigid registration approach for diffusion tensor images using a multicomponent information-theoretic measure. Explicit orientation optimization is enabled by incorporating tensor reorientation, which is necessary for wrapping diffusion tensor images. Experimental results on diffusion tensor images indicate the feasibility of the proposed approach and a much better performance compared to the affine registration method based on mutual information in terms of registration accuracy in the presence of geometric distortion.
This paper addresses the issue of matching rigid 3D object points with 2D image points through point registration based on maximum likelihood principle in computer simulated images. Perspective projection is necessary when transforming 3D coordinate into 2D. The problem then recasts into a missing data framework where unknown correspondences are handled via mixture models. Adopting the Expectation Conditional Maximization for Point Registration (ECMPR), two different rotation and translation optimization algorithms are compared in this paper. We analyze in detail the associated consequences in…
StyleSeg V2: Towards Robust One-shot Segmentation of Brain Tissue via Optimization-free Registration Error Perception
Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology（华中科技大学武汉光电国家研究中心）；Wuhan United Imaging Healthcare Surgical Technology Co., Ltd.（武汉联影外科技术有限公司）
University of Tehran（德黑兰大学）；Institute of Research in Fundamental Sciences (IPM)（基础科学研究所（IPM））
A viewing graph is a set of unknown camera poses, as the vertices, and the observed relative motions, as the edges. Solving the viewing graph is an essential step in a Structure-from-Motion procedure, where a set of relative motions is obtained from a collection of 2D images. Almost all methods in the literature solve for the rotations separately, through rotation averaging process, and use them for solving the positions. Obtaining positions is the challenging part because the translation observations only tell the direction of the motions. It becomes more challenging when the set of edges com…
XCalib Depth-Guided Geometric Optimization for Dense Thermal-Visible Video Registration
Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；CNRS（法国国家科学研究中心）；Grenoble INP–UGA（格勒诺布尔国立理工学院）；GIPSA-lab（GIPSA实验室）；Institut Universitaire de France (IUF)（法国大学研究院）；Lynred（Lynred公司）
