This is an empirical study to investigate the impact of scanner effects when using machine learning on multi-site neuroimaging data. We utilize structural T1-weighted brain MRI obtained from two different studies, Cam-CAN and UK Biobank. For the purpose of our investigation, we construct a dataset consisting of brain scans from 592 age- and sex-matched individuals, 296 subjects from each original study. Our results demonstrate that even after careful pre-processing with state-of-the-art neuroimaging pipelines a classifier can easily distinguish between the origin of the data with very high accuracy. Our analysis on the example application of sex classification suggests that current approaches to harmonize data are unable to remove scanner-specific bias leading to overly optimistic performance estimates and poor generalization. We conclude that multi-site data harmonization remains an open challenge and particular care needs to be taken when using such data with advanced machine learning methods for predictive modelling.
University of Basel（巴塞尔大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Gillberg Neuropsychiatry Center（吉尔伯格神经精神病学中心）；Sahlgrenska Academy（萨尔格伦斯卡学院）；University of Gothenburg（哥德堡大学）；University Hospital Basel（巴塞尔大学医院）
The limited availability of large image datasets, mainly due to data privacy and differences in acquisition protocols or hardware, is a significant issue in the development of accurate and generalizable machine learning methods in medicine. This is especially the case for Magnetic Resonance (MR) images, where different MR scanners introduce a bias that limits the performance of a machine learning model. We present a novel method that learns to ignore the scanner-related features present in MR images, by introducing specific additional constraints on the latent space. We focus on a real-world c…
Technical University of Munich（慕尼黑工业大学）；University Medical Center Freiburg（弗莱堡大学医学中心）
With the advent of deep learning algorithms, fully automated radiological image analysis is within reach. In spine imaging, several atlas- and shape-based as well as deep learning segmentation algorithms have been proposed, allowing for subsequent automated analysis of morphology and pathology. The first Large Scale Vertebrae Segmentation Challenge (VerSe 2019) showed that these perform well on normal anatomy, but fail in variants not frequently present in the training dataset. Building on that experience, we report on the largely increased VerSe 2020 dataset and results from the second iterat…
Deep Unsupervised Anomaly Detection in Brain Imaging: Large-Scale Benchmarking and Bias Analysis
Department of Psychiatry and Psychotherapy, University Hospital Tübingen（图宾根大学精神病学与心理学系）；Cluster of Excellence – Machine Learning for Science, University of Tübingen（图宾根大学机器学习科学卓越中心）；Department of Psychology, Friedrich Schiller University of Jena（耶拿弗里德里希·席勒大学心理学系）；German Center for Mental Health (DZPG), partner site Halle/Jena/Magdeburg（德国心理健康中心（DZPG）哈勒/耶拿/马格德堡分部）；University of Lucerne（卢塞恩大学）
Fighting the scanner effect in brain MRI segmentation with a progressive level-of-detail network trained on multi-site data
Center for Cognitive Neuroimaging（认知神经成像中心）；School of Psychology & Neuroscience（心理与神经科学学院）；University of Glasgow（格拉斯哥大学）；University of Brescia（布雷西亚大学）；Department of Information Engineering（信息工程系）
Imperial College London（帝国理工学院）；University College London（伦敦大学学院）；Barts Heart Centre, St Bartholomew’s Hospital（巴茨心脏中心，圣巴塞洛缪医院）；Queen Mary University of London（伦敦玛丽女王大学）
Convolutional neural network (CNN) based segmentation methods provide an efficient and automated way for clinicians to assess the structure and function of the heart in cardiac MR images. While CNNs can generally perform the segmentation tasks with high accuracy when training and test images come from the same domain (e.g. same scanner or site), their performance often degrades dramatically on images from different scanners or clinical sites. We propose a simple yet effective way for improving the network generalization ability by carefully designing data normalization and augmentation strateg…
Istituto Italiano di Tecnologia（意大利理工学院）；Simon Fraser University（西蒙弗雷泽大学）；University of Verona（维罗纳大学）；Fondation Campus Biotech（生物科技园区基金会）；University of British Columbia（不列颠哥伦比亚大学）；Fondazione Bruno Kessler（布鲁诺·凯斯勒基金会）
This paper presents a simple and effective generalization method for magnetic resonance imaging (MRI) segmentation when data is collected from multiple MRI scanning sites and as a consequence is affected by (site-)domain shifts. We propose to integrate a traditional encoder-decoder network with a regularization network. This added network includes an auxiliary loss term which is responsible for the reduction of the domain shift problem and for the resulting improved generalization. The proposed method was evaluated on multiple sclerosis lesion segmentation from MRI data. We tested the proposed…
The EOS(R) scanner is a radiographic system that captures PA and lateral images in standing posture. The system is widely used in diagnosis and assessment of scoliosis, as it provides a low-dose alternative to traditional X-ray and can capture full-body images. Furthermore, spacial calibration between the two imaging views is implemented in hardware, facilitating 3D reconstruction of imaging targets. In this paper, a brief description of the system is followed by an explanation of the geometric relationship between 3D space and radiographic image space.
University of Southern California（南加利福尼亚大学）；Information Sciences Institute（信息科学研究所）；Cardiff University（卡迪夫大学）
Purpose: In the present work we describe the correction of diffusion-weighted MRI for site and scanner biases using a novel method based on invariant representation. Theory and Methods: Pooled imaging data from multiple sources are subject to variation between the sources. Correcting for these biases has become very important as imaging studies increase in size and multi-site cases become more common. We propose learning an intermediate representation invariant to site/protocol variables, a technique adapted from information theory-based algorithmic fairness; by leveraging the data processing …
Harmonizing MR Images Across 100+ Scanners: Multi-site Validation with Traveling Subjects and Real-world Protocols
Johns Hopkins University（约翰斯·霍普金斯大学）；Vanderbilt University（范德堡大学）；University of Maryland School of Medicine（马里兰大学医学院）；National Institute on Aging（美国国家老龄化研究所）
Massachusetts Institute of Technology（麻省理工学院）；Information Sciences Institute（信息科学研究所）；Keck School of Medicine, University of Southern California（南加州大学凯克医学院）
Pooled imaging data from multiple sources is subject to bias from each source. Studies that do not correct for these scanner/site biases at best lose statistical power, and at worst leave spurious correlations in their data. Estimation of the bias effects is non-trivial due to the paucity of data with correspondence across sites, so called "traveling phantom" data, which is expensive to collect. Nevertheless, numerous solutions leveraging direct correspondence have been proposed. In contrast to this, Moyer et al. (2019) proposes an unsupervised solution using invariant representations, one whi…
Dual-energy computed tomography (DECT) is of great significance for clinical practice due to its huge potential to provide material-specific information. However, DECT scanners are usually more expensive than standard single-energy CT (SECT) scanners and thus are less accessible to undeveloped regions. In this paper, we show that the energy-domain correlation and anatomical consistency between standard DECT images can be harnessed by a deep learning model to provide high-performance DECT imaging from fully-sampled low-energy data together with single-view high-energy data, which can be obtaine…
Non-invasive detection of microvascular alterations in deep tissues in vivo provides critical information for clinical diagnosis and evaluation of a broad-spectrum of pathologies. Recently, the emergence of super-resolution ultrasound localization microscopy (ULM) offers new possibilities for clinical imaging of microvasculature at capillary level. Currently, the clinical utility of ULM on clinical ultrasound scanners is hindered by the technical limitations, such as long data acquisition time, and compromised tracking performance associated with low imaging frame-rate. Here we present an in-h…
A Portable Brain MRI Scanner for Underserved Settings and Point-Of-Care Imaging
Athinoula A. Martinos Center for Biomedical Imaging, Dept. of Radiology, Massachusetts General Hospital（阿西努拉·A·马丁诺斯生物医学成像中心放射科麻省总医院）；Harvard Medical School（哈佛医学院）；Dept. of Electrical Engineering, Massachusetts Institute of Technology（麻省理工学院电气工程系）；Dept. of Physics, Harvard University（哈佛大学物理系）；Dept. of Radiology, Massachusetts General Hospital（麻省总医院放射科）；Harvard-MIT Division of Health Sciences and Technology（哈佛-麻省理工健康科学与技术学部）
Magnetic Resonance Imaging (MRI) of hard biological tissues is challenging due to the fleeting lifetime and low strength of their response to resonant stimuli, especially at low magnetic fields. Consequently, the impact of MRI on some medical applications, such as dentistry, continues to be limited. Here, we present three-dimensional reconstructions of ex-vivo human teeth, as well as a rabbit head and part of a cow femur, all obtained at a field strength of only 260 mT. These images are the first featuring soft and hard tissues simultaneously at sub-Tesla fields, and they have been acquired in…
Purpose/Objectives: To develop and assess a strategy of using deep learning (DL) to generate virtual monochromatic CT (VMCT) images from a single-energy CT (SECT) scan. Materials/Methods: The proposed data-driven VMCT imaging consists of two steps: (i) using a supervised DL model trained with a large number of 100 kV and 140 kV dual-energy CT (DECT) image pairs to produce the corresponding high-energy CT image from a low-energy image; and (ii) reconstructing VMCT images with energy ranging from 40 to 150 keV. To evaluate the performance of the method, we retrospectively studied 6,767 abdominal…
The use of computed tomography (CT) imaging has become of increasing interest to academic areas outside of the field of medical imaging and industrial inspection, e.g., to biology and cultural heritage research. The pecularities of these fields, however, sometimes require that objects need to be imaged on-site, e.g., in field-work conditions or in museum collections. Under these circumstances, it is often not possible to use a commercial device and a custom solution is the only viable option. In order to achieve high image quality under adverse conditions, reliable calibration and trajectory r…
Background: The aim of this study was to assess the robustness of cardiac SPECT radiomics features against changes in imaging settings including acquisition and reconstruction settings. Methods: Four scanners were used to acquire SPECT scans of a cardiac phantom with 5mCi of 99mTc. The effects of different image acquisition and reconstruction settings including the Number of View, View Matrix Size, attenuation correction, image reconstruction algorithm, number of iterations, number of subsets, type of filter, full width at half maximum (FWHM) of Gaussian filter, Butterworth filter order, and B…
Image Sciences Institute, University Medical Center Utrecht（乌得勒支大学医学中心影像科学研究所）
Diffusion magnetic resonance imaging is a noninvasive imaging technique that can indirectly infer the microstructure of tissues and provide metrics which are subject to normal variability across subjects. Potentially abnormal values or features may yield essential information to support analysis of controls and patients cohorts, but subtle confounds affecting diffusion MRI, such as those due to difference in scanning protocols or hardware, can lead to systematic errors which could be mistaken for purely biologically driven variations amongst subjects. In this work, we propose a new harmonizati…
Deep learning (DL) can fail when there are data mismatches between training and testing data. Due to its operator-dependent nature, acquisition-related data mismatches, caused by different scanner settings, can occur in ultrasound imaging. Therefore, mitigating effects of such data mismatches is essential for wider clinical adoption of DL powered ultrasound imaging. To mitigate the effects, ideally we need to collect a large training set at each scanner setting. However, acquiring such training sets is expensive. Another approach could be training on a subset of imaging settings, which makes t…
LTCI, Télécom Paris, IP Paris（巴黎电信学院 LTCI，巴黎理工学院）；University of Turin（都灵大学）；NeuroSpin, CEA, Université Paris-Saclay（巴黎萨克雷大学原子能与替代能源委员会神经自旋中心）
Building accurate Deep Learning (DL) models for brain age prediction is a very relevant topic in neuroimaging, as it could help better understand neurodegenerative disorders and find new biomarkers. To estimate accurate and generalizable models, large datasets have been collected, which are often multi-site and multi-scanner. This large heterogeneity negatively affects the generalization performance of DL models since they are prone to overfit site-related noise. Recently, contrastive learning approaches have been shown to be more robust against noise in data or labels. For this reason, we pro…
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Northeastern University（东北大学）；Nanjing University of Science and Technology（南京理工大学）；Nanjing University（南京大学）；University of Virginia（弗吉尼亚大学）；Children’s National Medical Center（国立儿童医学中心）；Sungkyunkwan University（成均馆大学）；Southern Medical University（南方医科大学）；The Chinese University of Hong Kong（香港中文大学）；Shenzhen Institutes of Advanced Technology（深圳先进技术研究院）；Concordia University（康考迪亚大学）；Xiamen University（厦门大学）；Vanderbilt University（范德堡大学）；Stanford University（斯坦福大学）；Emory University（埃默里大学）；University of Jinan（济南大学）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能医疗科技有限公司）；Korea University（高丽大学）
To better understand early brain growth patterns in health and disorder, it is critical to accurately segment infant brain magnetic resonance (MR) images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF). Deep learning-based methods have achieved state-of-the-art performance; however, one of major limitations is that the learning-based methods may suffer from the multi-site issue, that is, the models trained on a dataset from one site may not be applicable to the datasets acquired from other sites with different imaging protocols/scanners. To promote methodological develo…
University of Kentucky（肯塔基大学）；University of Texas at Dallas（德克萨斯大学达拉斯分校）；UT Southwestern Medical Center（德克萨斯大学西南医学中心）；University of Texas Health Science Center at Houston（德克萨斯大学休斯顿健康科学中心）
While remarkable advances have been made in Computed Tomography (CT), most of the existing efforts focus on imaging enhancement while reducing radiation dose. How to harmonize CT image data captured using different scanners is vital in cross-center large-scale radiomics studies but remains the boundary to explore. Furthermore, the lack of paired training image problem makes it computationally challenging to adopt existing deep learning models. %developed for CT image standardization. %this problem more challenging. We propose a novel deep learning approach called CVH-CT for harmonizing CT imag…
Whole-Body Image-to-Image Translation for a Virtual Scanner in a Healthcare Digital Twin
Department of Computing Science, Umeå University（计算科学系，乌梅大学）；Department of Diagnostics and Intervention, Biomedical Engineering and Radiation Physics, Umeå University（诊断与介入系，生物医学工程与放射物理，乌梅大学）
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Harmonization improves data consistency and is central to effective integration of diverse imaging data acquired across multiple sites. Recent deep learning techniques for harmonization are predominantly supervised in nature and hence require imaging data of the same human subjects to be acquired at multiple sites. Data collection as such requires the human subjects to travel across sites and is hence challenging, costly, and impractical, more so when sufficient sample size is needed for reliable network training. Here we show how harmonization can be achieved with a deep neural network that d…
King’s College London（伦敦国王学院）；Guy’s and St Thomas’ NHS Foundation Trust（盖伊和圣托马斯国民保健服务基金会信托）；Technical University of Munich（慕尼黑工业大学）；Imperial College London（帝国理工学院）；Helmholtz Center Munich（慕尼黑亥姆霍兹中心）
Domain shift refers to the difference in the data distribution of two datasets, normally between the training set and the test set for machine learning algorithms. Domain shift is a serious problem for generalization of machine learning models and it is well-established that a domain shift between the training and test sets may cause a drastic drop in the model's performance. In medical imaging, there can be many sources of domain shift such as different scanners or scan protocols, different pathologies in the patient population, anatomical differences in the patient population (e.g. men vs wo…
Institute for Mathematical Innovation, University of Bath（巴斯大学数学创新研究所）
Imaging with multiple modalities or multiple channels is becoming increasingly important for our modern society. A key tool for understanding and early diagnosis of cancer and dementia is PET-MR, a combined positron emission tomography and magnetic resonance imaging scanner which can simultaneously acquire functional and anatomical data. Similarly in remote sensing, while hyperspectral sensors may allow to characterise and distinguish materials, digital cameras offer high spatial resolution to delineate objects. In both of these examples, the imaging modalities can be considered individually o…
The ability to achieve submillimter isotropic resolution diffusion MR imaging (dMRI) is critically important to study fine-scale brain structures, particularly in the cortex. One of the major challenges in performing submillimeter dMRI is the inherently low signal-to-noise ratio (SNR). While approaches capable of mitigating the low SNR in high resolution dMRI have been proposed, namely simultaneous multi-slab (SMSlab) and generalized slice dithered enhanced resolution with simultaneous multislice (gSlider-SMS), limitations are associated with these approaches. The SMSlab sequences suffer from …
A central task in medical imaging is the reconstruction of an image or function from data collected by medical devices (e.g., CT, MRI, and PET scanners). We provide quantum algorithms for image reconstruction with exponential speedup over classical counterparts when data is input as a quantum state. Since outputs of our algorithms are stored in quantum states, individual pixels of reconstructed images may not be efficiently accessed classically; instead, we discuss various methods to extract information from outputs using a variety of quantum post-processing algorithms.
In a standard computed tomography (CT) image, pixels having the same Hounsfield Units (HU) can correspond to different materials and it is therefore challenging to differentiate and quantify materials. Dual-energy CT (DECT) is desirable to differentiate multiple materials, but DECT scanners are not widely available as single-energy CT (SECT) scanners. Here we purpose a deep learning approach to perform DECT imaging by using standard SECT data. We designed a predenoising and difference learning mechanism to generate DECT images from SECT data. The performance of the deep learning-based DECT app…
Quality assessment of brain structural MR images: Comparing generalization of deep learning versus hand-crafted feature-based machine learning methods to new sites
Hawkes Institute and Department of Computer Science, University College London（伦敦大学学院）；Neuroradiological Academic Unit, UCL Queen Square Institute of Neurology, University College London（伦敦大学学院）；Queen Square Centre for Neuromuscular Diseases, Department of Neuromuscular Diseases, UCL Queen Square Institute of Neurology（伦敦大学学院）；Radiology & Nuclear Medicine, VU University Medical Center（阿姆斯特丹自由大学医学中心）
