Registration by Regression (RbR): a framework for interpretable and flexible atlas registration
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Danish Research Centre for Magnetic Resonance（丹麦磁共振研究中心）；Copenhagen University Hospital（哥本哈根大学医院）；Centre for Medical Image Computing（医学图像计算中心）；University College London（伦敦大学学院）；Computer Science and AI Laboratory（计算机科学与人工智能实验室）；Massachusetts Institute of Technology（麻省理工学院）
In human neuroimaging studies, atlas registration enables mapping MRI scans to a common coordinate frame, which is necessary to aggregate data from multiple subjects. Machine learning registration methods have achieved excellent speed and accuracy but lack interpretability and flexibility at test time (since their deformation model is fixed). More recently, keypoint-based methods have been proposed to tackle these issues, but their accuracy is still subpar, particularly when fitting nonlinear transforms. Here we propose Registration by Regression (RbR), a novel atlas registration framework that: is highly robust and flexible; can be trained with cheaply obtained data; and operates on a single channel, such that it can also be used as pretraining for other tasks. RbR predicts the (x, y, z) atlas coordinates for every voxel of the input scan (i.e., every voxel is a keypoint), and then uses closed-form expressions to quickly fit transforms using a wide array of possible deformation models, including affine and nonlinear (e.g., Bspline, Demons, invertible diffeomorphic models, etc.). Robustness is provided by the large number of voxels informing the registration and can be further increased by robust estimators like RANSAC. Experiments on independent public datasets show that RbR yields more accurate registration than competing keypoint approaches, over a wide range of deformation models.
East China Normal University（华东师范大学）；University of Virginia（弗吉尼亚大学）；Shanghai University（上海大学）
Registration-based atlas building often poses computational challenges in high-dimensional image spaces. In this paper, we introduce a novel hybrid atlas building algorithm that fast estimates atlas from large-scale image datasets with much reduced computational cost. In contrast to previous approaches that iteratively perform registration tasks between an estimated atlas and individual images, we propose to use learned priors of registration from pre-trained neural networks. This newly developed hybrid framework features several advantages of (i) providing an efficient way of atlas building w…
Purpose: Segmentation of organs-at-risk (OARs) is a bottleneck in current radiation oncology pipelines and is often time consuming and labor intensive. In this paper, we propose an atlas-based semi-supervised registration algorithm to generate accurate segmentations of OARs for which there are ground truth contours and rough segmentations of all other OARs in the atlas. To the best of our knowledge, this is the first study to use learning-based registration methods for the segmentation of head and neck patients and demonstrate its utility in clinical applications. Methods: Our algorithm cascad…
Fuzhou University（福州大学）；Fudan University（复旦大学）；Shanghai Jiao Tong University（上海交通大学）；King’s College London（伦敦国王学院）
Both image registration and label fusion in the multi-atlas segmentation (MAS) rely on the intensity similarity between target and atlas images. However, such similarity can be problematic when target and atlas images are acquired using different imaging protocols. High-level structure information can provide reliable similarity measurement for cross-modality images when cooperating with deep neural networks (DNNs). This work presents a new MAS framework for cross-modality images, where both image registration and label fusion are achieved by DNNs. For image registration, we propose a consiste…
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Atlas building and image registration are important tasks for medical image analysis. Once one or multiple atlases from an image population have been constructed, commonly (1) images are warped into an atlas space to study intra-subject or inter-subject variations or (2) a possibly probabilistic atlas is warped into image space to assign anatomical labels. Atlas estimation and nonparametric transformations are computationally expensive as they usually require numerical optimization. Additionally, previous approaches for atlas building often define similarity measures between a fuzzy atlas and …
CEA, Fundamental Research Division (DRF), Institute Jacob, Molecular Imaging Research Center (MIRCen)（法国原子能与可替代能源委员会（CEA）基础研究部雅各布研究所分子影像研究中心（MIRCen））；WITSEE（WITSEE 公司）
A significant challenge for brain histological data analysis is to precisely identify anatomical regions in order to perform accurate local quantifications and evaluate therapeutic solutions. Usually, this task is performed manually, becoming therefore tedious and subjective. Another option is to use automatic or semi-automatic methods, among which segmentation using digital atlases co-registration. However, most available atlases are 3D, whereas digitized histological data are 2D. Methods to perform such 2D-3D segmentation from an atlas are required. This paper proposes a strategy to automati…
Deep learning models for semantic segmentation are able to learn powerful representations for pixel-wise predictions, but are sensitive to noise at test time and do not guarantee a plausible topology. Image registration models on the other hand are able to warp known topologies to target images as a means of segmentation, but typically require large amounts of training data, and have not widely been benchmarked against pixel-wise segmentation models. We propose Atlas-ISTN, a framework that jointly learns segmentation and registration on 2D and 3D image data, and constructs a population-derived…
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Erasmus MC - University Medical Center Rotterdam（伊拉斯谟医学中心——鹿特丹大学医学中心）；University of Copenhagen（哥本哈根大学）；Massachusetts Institute of Technology（麻省理工学院）
Registration is a core component of many imaging pipelines. In case of clinical scans, with lower resolution and sometimes substantial motion artifacts, registration can produce poor results. Visual assessment of registration quality in large clinical datasets is inefficient. In this work, we propose to automatically assess the quality of registration to an atlas in clinical FLAIR MRI scans of the brain. The method consists of automatically segmenting the ventricles of a given scan using a neural network, and comparing the segmentation to the atlas' ventricles propagated to image space. We use…
Multi-atlas segmentation (MAS) is a popular image segmentation technique for medical images. In this work, we improve the performance of MAS by correcting registration errors before label fusion. Specifically, we use a volumetric displacement field to refine registrations based on image anatomical appearance and predicted labels. We show the influence of the initial spatial alignment as well as the beneficial effect of using label information for MAS performance. Experiments demonstrate that the proposed refinement approach improves MAS performance on a 3D magnetic resonance dataset of the kne…
Multi-atlas segmentation (MAS) is a promising framework for medical image segmentation. Generally, MAS methods register multiple atlases, i.e., medical images with corresponding labels, to a target image; and the transformed atlas labels can be combined to generate target segmentation via label fusion schemes. Many conventional MAS methods employed the atlases from the same modality as the target image. However, the number of atlases with the same modality may be limited or even missing in many clinical applications. Besides, conventional MAS methods suffer from the computational burden of reg…
Wuhan University（武汉大学）；Rensselaer Polytechnic Institute (RPI)（伦斯勒理工学院（RPI））
Medical image segmentation is a fundamental task in medical image analysis. Despite that deep convolutional neural networks have gained stellar performance in this challenging task, they typically rely on large labeled datasets, which have limited their extension to customized applications. By revisiting the superiority of atlas based segmentation methods, we present a new framework of One-pass aligned Atlas Set for Images Segmentation (OASIS). To address the problem of time-consuming iterative image registration used for atlas warping, the proposed method takes advantage of the power of deep …
University College London（伦敦大学学院）；Inria（法国国家信息与自动化研究所）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Massachusetts Institute of Technology（麻省理工学院）
Joint registration of a stack of 2D histological sections to recover 3D structure (``3D histology reconstruction'') finds application in areas such as atlas building and validation of \emph{in vivo} imaging. Straightforward pairwise registration of neighbouring sections yields smooth reconstructions but has well-known problems such as ``banana effect'' (straightening of curved structures) and ``z-shift'' (drift). While these problems can be alleviated with an external, linearly aligned reference (e.g., Magnetic Resonance (MR) images), registration is often inaccurate due to contrast difference…
GE Healthcare（通用电气医疗）；MAP5, Universite de Paris（巴黎大学MAP5实验室）
Automatic annotation of anatomical structures can help simplify workflow during interventions in numerous clinical applications but usually involves a large amount of annotated data. The complexity of the labeling task, together with the lack of representative data, slows down the development of robust solutions. In this paper, we propose a solution requiring very few annotated cases to label 3D pelvic arterial trees of patients with benign prostatic hyperplasia. We take advantage of Large Deformation Diffeomorphic Metric Mapping (LDDMM) to perform registration based on meaningful deformations…
UNC Chapel Hill（北卡罗来纳大学教堂山分校）；University of Pennsylvania（宾夕法尼亚大学）
Registration of images with pathologies is challenging due to tissue appearance changes and missing correspondences caused by the pathologies. Moreover, mass effects as observed for brain tumors may displace tissue, creating larger deformations over time than what is observed in a healthy brain. Deep learning models have successfully been applied to image registration to offer dramatic speed up and to use surrogate information (e.g., segmentations) during training. However, existing approaches focus on learning registration models using images from healthy patients. They are therefore not desi…
StyleSeg V2: Towards Robust One-shot Segmentation of Brain Tissue via Optimization-free Registration Error Perception
Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology（华中科技大学武汉光电国家研究中心）；Wuhan United Imaging Healthcare Surgical Technology Co., Ltd.（武汉联影外科技术有限公司）
We propose an unsupervised deep learning method for atlas based registration to achieve segmentation and spatial alignment of the embryonic brain in a single framework. Our approach consists of two sequential networks with a specifically designed loss function to address the challenges in 3D first trimester ultrasound. The first part learns the affine transformation and the second part learns the voxelwise nonrigid deformation between the target image and the atlas. We trained this network end-to-end and validated it against a ground truth on synthetic datasets designed to resemble the challen…
JOSA: Joint surface-based registration and atlas construction of brain geometry and function
McGovern Institute for Brain Research, MIT（麦戈文脑科学研究所（麻省理工学院））；Harvard University（哈佛大学）；Computer Science and Artificial Intelligence Laboratory, MIT（计算机科学与人工智能实验室（麻省理工学院））
UCSD（加州大学圣地亚哥分校）；Salk Institute for Biological Studies（索尔克生物研究所）
Brightfield and fluorescent imaging of whole brain sections are funda- mental tools of research in mouse brain study. As sectioning and imaging become more efficient, there is an increasing need to automate the post-processing of sec- tions for alignment and three dimensional visualization. There is a further need to facilitate the development of a digital atlas, i.e. a brain-wide map annotated with cell type and tract tracing data, which would allow the automatic registra- tion of images stacks to a common coordinate system. Currently, registration of slices requires manual identification of …
CLARITY is a method for converting biological tissues into translucent and porous hydrogel-tissue hybrids. This facilitates interrogation with light sheet microscopy and penetration of molecular probes while avoiding physical slicing. In this work, we develop a pipeline for registering CLARIfied mouse brains to an annotated brain atlas. Due to the novelty of this microscopy technique it is impractical to use absolute intensity values to align these images to existing standard atlases. Thus we adopt a large deformation diffeomorphic approach for registering images via mutual information matchin…
Unsupervised Segmentation of Fetal Brain MRI using Deep Learning Cascaded Registration
Universitat Pompeu Fabra（庞培法布拉大学）；BCNatal；Institución Catalana de Investigación y Estudios Avanzados (ICREA)（加泰罗尼亚高级研究与发展研究院）；Centre for Biomedical Research on Rare Diseases (CIBERER)（罕见疾病生物医学研究中心）；European Commission, Joint Research Centre (JRC)（欧盟委员会联合研究中心）
Brain Research Institute (HiFo), UZH（苏黎世大学脑研究所（HiFo））；Neuroscience Center Zurich (ZNZ), UZH/ETH Zurich（苏黎世神经科学中心（ZNZ），苏黎世大学/苏黎世联邦理工学院）；Department of Biology (D-BIOL), ETH Zurich（苏黎世联邦理工学院生物系（D-BIOL））
Neuroscientists have devoted significant effort into the creation of standard brain reference atlases for high-throughput registration of anatomical regions of interest. However, variability in brain size and form across individuals poses a significant challenge for such reference atlases. To overcome these limitations, we introduce a fully automated deep neural network-based method (SeBRe) for registration through Segmenting Brain Regions of interest with minimal human supervision. We demonstrate the validity of our method on brain images from different mouse developmental time points, across…
Vanderbilt University（范德堡大学）；Vanderbilt University Medical Center（范德堡大学医学中心）；U.S. Department of Veterans Affairs（美国退伍军人事务部）；University of Lübeck（吕贝克大学）
A major goal of lung cancer screening is to identify individuals with particular phenotypes that are associated with high risk of cancer. Identifying relevant phenotypes is complicated by the variation in body position and body composition. In the brain, standardized coordinate systems (e.g., atlases) have enabled separate consideration of local features from gross/global structure. To date, no analogous standard atlas has been presented to enable spatial mapping and harmonization in chest computational tomography (CT). In this paper, we propose a thoracic atlas built upon a large low dose CT …
Deep-Learning Atlas Registration for Melanoma Brain Metastases: Preserving Pathology While Enabling Cohort-Level Analyses
Department of Radiation Oncology, Medical Center – University of Freiburg, Faculty of Medicine, University of Freiburg（放射肿瘤科，弗赖堡大学医学中心-弗赖堡大学医学院）；Division of Medical Physics, Department of Radiation Oncology, Medical Center – University of Freiburg, Faculty of Medicine, University of Freiburg（医学物理系，放射肿瘤科，弗赖堡大学医学中心-弗赖堡大学医学院）；Department of Radiation Oncology, University Medical Center Schleswig-Holstein（放射肿瘤科，石勒苏益格-荷尔斯泰因大学医学中心）；Department of Radiation Oncology, TUM School of Medicine and Health, TUM University Hospital Rechts der Isar, Technical University of Munich（放射肿瘤科，慕尼黑技术大学医学院及健康科学系，慕尼黑技术大学医院雷chts der Isar）
Murdoch University（莫道克大学）；Harry Butler Institute, Murdoch University（莫道克大学哈里·巴特勒研究所）；University of South Australia（南澳大学）；TU Berlin（柏林工业大学）；Durham University（杜伦大学）；Ohio State University（俄亥俄州立大学）；University of Western Australia（西澳大学）；Florida State University（佛罗里达州立大学）
We propose a novel framework to learn the spatiotemporal variability in longitudinal 3D shape data sets, which contain observations of objects that evolve and deform over time. This problem is challenging since surfaces come with arbitrary parameterizations and thus, they need to be spatially registered. Also, different deforming objects, also called 4D surfaces, evolve at different speeds and thus they need to be temporally aligned. We solve this spatiotemporal registration problem using a Riemannian approach. We treat a 3D surface as a point in a shape space equipped with an elastic Riemanni…
JParc: Joint cortical surface parcellation with registration
1 Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, MGH \& HMS 2 Center for Neurotechnology；Neurorecovery, Department of Neurology, MGH \& HMS 3 Computer Science；Artificial Intelligence Laboratory, EECS, MIT
Accurate and robust whole heart substructure segmentation is crucial in developing clinical applications, such as computer-aided diagnosis and computer-aided surgery. However, segmentation of different heart substructures is challenging because of inadequate edge or boundary information, the complexity of the background and texture, and the diversity in different substructures' sizes and shapes. This article proposes a framework for multi-class whole heart segmentation employing non-rigid registration-based probabilistic atlas incorporating the Bayesian framework. We also propose a non-rigid r…
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Biomedical Research Imaging Center (BRIC)（生物医学研究成像中心）；UNC Chapel Hill（北卡罗来纳大学教堂山分校）；University of Salzburg（萨尔茨堡大学）
This paper introduces Quicksilver, a fast deformable image registration method. Quicksilver registration for image-pairs works by patch-wise prediction of a deformation model based directly on image appearance. A deep encoder-decoder network is used as the prediction model. While the prediction strategy is general, we focus on predictions for the Large Deformation Diffeomorphic Metric Mapping (LDDMM) model. Specifically, we predict the momentum-parameterization of LDDMM, which facilitates a patch-wise prediction strategy while maintaining the theoretical properties of LDDMM, such as guaranteed…
CTARR: A fast and robust method for identifying anatomical regions on CT images via atlas registration
University Medical Center Hamburg-Eppendorf（汉堡埃彭多夫大学医学中心）；jung diagnostics（jung诊断公司）；University of Cambridge（剑桥大学）；Radboud University Medical Center（拉德堡德大学医学中心）；Early Cancer Institute（早期癌症研究所）；Cancer Research UK Cambridge Centre（英国癌症研究剑桥中心）
AI-assisted surgeries have drawn the attention of the medical image research community due to their real-world impact on improving surgery success rates. For image-guided surgeries, such as Cochlear Implants (CIs), accurate object segmentation can provide useful information for surgeons before an operation. Recently published image segmentation methods that leverage machine learning usually rely on a large number of manually predefined ground truth labels. However, it is a laborious and time-consuming task to prepare the dataset. This paper presents a novel technique using a self-supervised 3D…
Institute of Medical Informatics, Universität zu Lübeck（吕贝克大学医学信息学研究所）
Nonlinear image registration continues to be a fundamentally important tool in medical image analysis. Diagnostic tasks, image-guided surgery and radiotherapy as well as motion analysis all rely heavily on accurate intra-patient alignment. Furthermore, inter-patient registration enables atlas-based segmentation or landmark localisation and shape analysis. When labelled scans are scarce and anatomical differences large, conventional registration has often remained superior to deep learning methods that have so far mainly dealt with relatively small or low-complexity deformations. We address thi…
FractMorph: A Fractional Fourier-Based Multi-Domain Transformer for Deformable Image Registration
Faculty of Computer Science and Engineering, Shahid Beheshti University（谢赫·贝赫什特大学计算机科学与工程学院）；School of Computing Science, University of Glasgow（格拉斯哥大学计算机科学学院）；Alan Turing Institute（艾伦·图灵研究所）
Reference-Free 3D Reconstruction of Brain Dissection Slabs via Learned Atlas Coordinates
Martinos Center for Biomedical Imaging (Martinos Center) at Massachusetts General Hospital (MGH) & Harvard Medical School (HMS)（马萨诸塞州总医院（MGH）及哈佛医学院（HMS）的生物医学成像中心（Martinos Center））；Computer Science and Artificial Intelligence Laboratory (CSAIL) at the Massachusetts Institute of Technology (MIT)（麻省理工学院（MIT）的计算机科学与人工智能实验室（CSAIL））；Danish Research Centre for Magnetic Resonance, Centre for Functional and Diagnostic Imaging and Research, Copenhagen University Hospital-Amager and Hvidovre, Copenhagen, Denmark（丹麦磁共振研究中心、功能与诊断成像及研究中心，哥本哈根大学医院-阿迈厄斯和赫维多尔，哥本哈根，丹麦）；Massachusetts Alzheimer’s Disease Research Center at MGH & HMS（马萨诸塞州总医院（MGH）及哈佛医学院（HMS）的阿尔茨海默病研究中心）；University of Washington（华盛顿大学）；Pathology Department at MGH & HMS（马萨诸塞州总医院（MGH）及哈佛医学院（HMS）的病理部门）；Neurology Department at MGH & HMS（马萨诸塞州总医院（MGH）及哈佛医学院（HMS）的神经病学部门）；Department of（部门）
BrainLesion Suite: A Flexible and User-Friendly Framework for Modular Brain Lesion Image Analysis
Department of Quantitative Biomedicine, University of Zurich, Zurich, Switzerland（苏黎世大学定量生物医学系）；Helmholtz AI, Helmholtz Zentrum München, Germany（海德堡人工智能，慕尼黑研究中心）；Department of Diagnostic and Interventional Neuroradiology, School of Medicine, Klinikum rechts der Isar, Technical University of Munich, Germany（医学诊断与介入神经放射学系，慕尼黑技术大学）；TranslaTUM - Central Institute for Translational Cancer Research, Technical University of Munich, Germany（TranslaTUM - 转化癌症研究中央研究所，慕尼黑技术大学）；Chair for AI in Medicine, Klinikum Rechts der Isar, Munich, Germany（医学人工智能系，慕尼黑Klinikum Rechts der Isar）；Department of Radiation Oncology, Klinikum rechts der Isar, Technical University of Munich, Germany（放射肿瘤学系，慕尼黑Klinikum Rechts der Isar）；Institute of Clinical Neuroimmunology, Faculty of Medicine, Ludwig Maximilian University, Munich, Germany（临床神经免疫学研究所，路德维希-马克西米利安大学）；School of Computation, Information and Technology, Technical University of Munich, Germany（计算、信息与技术学院，慕尼黑技术大学）；Division of Computational Pathology, Indiana School of Medicine, Indianapolis, IN, USA（计算病理学系，印第安纳医学院）；Research Center for Digital Technologies in Dentistry and CAD/CAM, Department of Dentistry, Faculty of Medicine and Dentistry, Danube Private University, Krems, Austria（牙科数字技术研究中心，医学与牙科学院，多瑙私人大学）；Medical Research Group, MLCommons, San Francisco, CA, USA（医学研究组，MLCommons，旧金山，美国）；Department of Neurology, Clinical Neuroscience Center and Brain Tumor Center, University and University Hospital Zurich, 8091 Zurich, Switzerland（神经学系，临床神经科学中心和脑肿瘤中心，苏黎世大学及苏黎世大学医院）；Department of Medical Radiation Physics, Stockholm University, Solna, Sweden（医学辐射物理学系，斯德哥尔摩大学）；Department of Oncology-Pathology, Karolinska Institutet, Solna, Sweden, Sweden（肿瘤病理学系，卡罗林斯卡研究所）；Division of Oncology and Children's Research Center, University Children's Hospital Zurich, Switzerland（肿瘤与儿童研究中心，苏黎世儿童医院）；Wallace H. Coulter Department of Biomedical Engineering at Georgia Tech and Emory University, Atlanta, USA（Wallace H. Coulter生物医学工程系，佐治亚理工学院和埃默里大学，美国）；Institute of Radiation Medicine (IRM), Helmholtz Zentrum München (HMGU), Germany（放射医学研究所（IRM），海德堡研究中心（HMGU））；German Consortium for Translational Cancer Research (DKTK), Partner Site Munich, Munich, Germany（德国转化癌症研究联盟（DKTK），慕尼黑合作伙伴站点）
