FIRE: Unsupervised bi-directional inter-modality registration using deep networks
University of Edinburgh（爱丁堡大学）；Sun Yat-Sen University（中山大学）
Chengjia Wang, Giorgos Papanastasiou, Agisilaos Chartsias, Grzegorz Jacenkow, Sotirios A. Tsaftaris, Heye Zhang
备注：
We submitted this paper to a top medical imaging conference, srebuttal responded by the meta-reviewer. We were told that this work is not important and will not have big impact as the "reviewers were not enthusiastic". Here I publish the paper online for an open discussion. I will publish the code, the pre-trained model, the results, especially the reviews, and the meta reviews on github
Inter-modality image registration is an critical preprocessing step for many applications within the routine clinical pathway. This paper presents an unsupervised deep inter-modality registration network that can learn the optimal affine and non-rigid transformations simultaneously. Inverse-consistency is an important property commonly ignored in recent deep learning based inter-modality registration algorithms. We address this issue through the proposed multi-task architecture and the new comprehensive transformation network. Specifically, the proposed model learns a modality-independent latent representation to perform cycle-consistent cross-modality synthesis, and use an inverse-consistent loss to learn a pair of transformations to align the synthesized image with the target. We name this proposed framework as FIRE due to the shape of its structure. Our method shows comparable and better performances with the popular baseline method in experiments on multi-sequence brain MR data and intra-modality 4D cardiac Cine-MR data.
Factored Levenberg-Marquardt for Diffeomorphic Image Registration: An efficient optimizer for FireANTs
Department of Computer and Information Science（计算机与信息科学系）；University of Pennsylvania（宾夕法尼亚大学）；Department of Electrical and Systems Engineering（电气与系统工程系）；Department of Radiology & Penn Image Computing and Science Laboratory (PICSL)（放射科及宾夕法尼亚大学图像计算与科学实验室（PICSL））；Perelman School of Medicine, University of Pennsylvania（佩尔曼医学学院，宾夕法尼亚大学）
Queensland University of Technology（昆士兰科技大学）；CSIRO（联邦科学与工业研究组织）
The recent application of deep learning technologies in medical image registration has exponentially decreased the registration time and gradually increased registration accuracy when compared to their traditional counterparts. Most of the learning-based registration approaches considers this task as a one directional problem. As a result, only correspondence from the moving image to the target image is considered. However, in some medical procedures bidirectional registration is required to be performed. Unlike other learning-based registration, we propose a registration framework with invers…
Bi-directional Self-Registration for Misaligned Infrared-Visible Image Fusion
College of Intelligence and Computing（智能与计算学院）；Tianjin University（天津大学）；School of Computer Science and Technology（计算机科学与技术学校）；Chongqing University of Posts and Telecommunications（重庆邮电大学）
We present INSPIRE, a top-performing general-purpose method for deformable image registration. INSPIRE brings distance measures which combine intensity and spatial information into an elastic B-splines-based transformation model and incorporates an inverse inconsistency penalization supporting symmetric registration performance. We introduce several theoretical and algorithmic solutions which provide high computational efficiency and thereby applicability of the proposed framework in a wide range of real scenarios. We show that INSPIRE delivers highly accurate, as well as stable and robust reg…
ConKeD++ -- Improving descriptor learning for retinal image registration: A comprehensive study of contrastive losses
Instituto de Investigacion Biomédica de A Coruña (INIBIC)（拉科鲁尼亚生物医学研究所）；Universidade da Coruña（拉科鲁尼亚大学）；Departamento de Ciencias de la Computación y Tecnologías de la Información（计算机科学与信息技术系）；Grupo VARPA（VARPA团队）
University of Edinburgh（爱丁堡大学）；Imperial College London（帝国理工学院）；University of Essex（埃塞克斯大学）
Magnetic Resonance Imaging (MRI) typically recruits multiple sequences (defined here as "modalities"). As each modality is designed to offer different anatomical and functional clinical information, there are evident disparities in the imaging content across modalities. Inter- and intra-modality affine and non-rigid image registration is an essential medical image analysis process in clinical imaging, as for example before imaging biomarkers need to be derived and clinically evaluated across different MRI modalities, time phases and slices. Although commonly needed in real clinical scenarios, …
Multi-atlas segmentation (MAS) is a promising framework for medical image segmentation. Generally, MAS methods register multiple atlases, i.e., medical images with corresponding labels, to a target image; and the transformed atlas labels can be combined to generate target segmentation via label fusion schemes. Many conventional MAS methods employed the atlases from the same modality as the target image. However, the number of atlases with the same modality may be limited or even missing in many clinical applications. Besides, conventional MAS methods suffer from the computational burden of reg…
Hyperspectral imaging is a cutting-edge type of remote sensing used for mapping vegetation properties, rock minerals and other materials. A major drawback of hyperspectral imaging devices is their intrinsic low spatial resolution. In this paper, we propose a method for increasing the spatial resolution of a hyperspectral image by fusing it with an image of higher spatial resolution that was obtained with a different imaging modality. This is accomplished by solving a variational problem in which the regularization functional is the directional total variation. To accommodate for possible mis-r…
Progressive Retinal Image Registration via Global and Local Deformable Transformations
School of Computer Science Wuhan University Wuhan, China；School of Computer Science The University of Sydney Sydney, Australia；School of Computer Science Wuhan University Wuhan, China yuliang\；Institute for Infocomm Research A STAR Singapore cheng\
Multi-Expert Learning Framework with the State Space Model for Optical and SAR Image Registration
Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education of China（教育部智能感知与图像理解重点实验室）；Hangzhou Institute of Technology, Xidian University（西安电子科技大学杭州学院）；School of Computer Science, Xidian University（西安电子科技大学计算机学院）；Department of Automation, Tsinghua University（清华大学自动化系）
MICDIR: Multi-scale Inverse-consistent Deformable Image Registration using UNetMSS with Self-Constructing Graph Latent
Otto von Guericke University Magdeburg（马格德堡奥托·冯·格里克大学）；Human Technopole（人类技术园）；German Centre for Neurodegenerative Disease（德国神经退行性疾病中心）；Centre for Behavioural Brain Sciences（行为脑科学中心）
School of Computer Science and Statistics（计算机科学与统计学院）；Trinity College Dublin（都柏林大学三一学院）
We propose several cost functions for registration of shapes encoded with Euclidean and/or non-Euclidean information (unit vectors). Our framework is assessed for estimation of both rigid and non-rigid transformations between the target and model shapes corresponding to 2D contours and 3D surfaces. The experimental results obtained confirm that using the combination of a point's position and unit normal vector in a cost function can enhance the registration results compared to state of the art methods.
This paper presents a robust probabilistic point registration method for estimating the rigid transformation (i.e. rotation matrix and translation vector) between two pointcloud dataset. The method improves the robustness of point registration and consequently the robot localization in the presence of outliers in the pointclouds which always occurs due to occlusion, dynamic objects, and sensor errors. The framework models the point registration task based on directional statistics on a unit sphere. In particular, a Kent distribution mixture model is adopted and the process of point registratio…
In image registration, many efforts have been devoted to the development of alternatives to the popular normalized mutual information criterion. Concurrently to these efforts, an increasing number of works have demonstrated that substantial gains in registration accuracy can also be achieved by aligning structural representations of images rather than images themselves. Following this research path, we propose a new method for mono- and multimodal image registration based on the alignment of regularized vector fields derived from structural information such as gradient vector flow fields, a te…
We present an information-theoretic approach to the registration of images with directional information, and especially for diffusion-Weighted Images (DWI), with explicit optimization over the directional scale. We call it Locally Orderless Registration with Directions (LORD). We focus on normalized mutual information as a robust information-theoretic similarity measure for DWI. The framework is an extension of the LOR-DWI density-based hierarchical scale-space model that varies and optimizes the integration, spatial, directional, and intensity scales. As affine transformations are insufficien…
Charles Sturt University（查尔斯特大学）；Study Group Australia（澳大利亚学习集团）；University of Technology, Baghdad（巴格达技术大学）；University of Kurdistan Hewler（库尔德斯坦赫勒大学）
Background and aim: Image registration and alignment are the main limitations of augmented reality-based knee replacement surgery. This research aims to decrease the registration error, eliminate outcomes that are trapped in local minima to improve the alignment problems, handle the occlusion, and maximize the overlapping parts. Methodology: markerless image registration method was used for Augmented reality-based knee replacement surgery to guide and visualize the surgical operation. While weight least square algorithm was used to enhance stereo camera-based tracking by filling border occlusi…
The non-rigid registration between CT data and ultrasonic images of liver can facilitate the diagnosis and treatment, which has been widely studied in recent years. To improve the registration accuracy of the Demons model on the non-rigid registration between 3D CT liver data and 2D ultrasonic images, a novel boundary extraction and enhancement method based on radial directional local intuitionistic fuzzy entropy in the polar coordinates has been put forward, and a new registration workflow has been provided. Experiments show that our method can acquire high-accuracy registration results. Expe…
Dense real-time tracking and mapping from RGB-D images is an important tool for many robotic applications, such as navigation and manipulation. The recently presented Directional Truncated Signed Distance Function (DTSDF) is an augmentation of the regular TSDF that shows potential for more coherent maps and improved tracking performance. In this work, we present methods for rendering depth- and color images from the DTSDF, making it a true drop-in replacement for the regular TSDF in established trackers. We evaluate the algorithm on well-established datasets and observe that our method improve…
FIRe: Fast Inverse Rendering using Directional and Signed Distance Functions
TU Munich（慕尼黑工业大学）；MCML（慕尼黑计算数学与机器学习实验室）；Massachusetts Institute of Technology（麻省理工学院）；University of Bonn（波恩大学）；MPI Informatics（马克斯·普朗克信息学研究所）；SIC（萨尔信息中心）
Registration is a Representation Problem: Solving Whole-Body PET/CT Registration
Martinos Center for Biomedical Imaging, Massachusetts General Hospital（麻省总医院马蒂诺斯生物医学成像中心）；Harvard Medical School（哈佛医学院）；Computer Science and AI Lab, Massachusetts Institute of Technology（麻省理工学院计算机科学与人工智能实验室）
FireANTs: Adaptive Riemannian Optimization for Multi-Scale Diffeomorphic Matching
Computer and Information Science, University of Pennsylvania（宾夕法尼亚大学计算机与信息科学系）；Penn Image Computing and Science Laboratory, University of Pennsylvania（宾夕法尼亚大学宾夕法尼亚图像计算与科学实验室）；Electrical and Systems Engineering, University of Pennsylvania（宾夕法尼亚大学电子与系统工程系）；Radiology, Perelman School of Medicine, University of Pennsylvania（宾夕法尼亚大学放射学系）
Rutgers University（罗格斯大学）；New York University School of Medicine（纽约大学医学院）；NVIDIA（英伟达）；SenseBrain；Shanghai AI Laboratory（上海人工智能实验室）
Cardiac tagging magnetic resonance imaging (t-MRI) is the gold standard for regional myocardium deformation and cardiac strain estimation. However, this technique has not been widely used in clinical diagnosis, as a result of the difficulty of motion tracking encountered with t-MRI images. In this paper, we propose a novel deep learning-based fully unsupervised method for in vivo motion tracking on t-MRI images. We first estimate the motion field (INF) between any two consecutive t-MRI frames by a bi-directional generative diffeomorphic registration neural network. Using this result, we then e…
