A significant proportion of patients scanned in a clinical setting have follow-up scans. We show in this work that such longitudinal scans alone can be used as a form of 'free' self-supervision for training a deep network. We demonstrate this self-supervised learning for the case of T2-weighted sagittal lumbar Magnetic Resonance Images (MRIs). A Siamese convolutional neural network (CNN) is trained using two losses: (i) a contrastive loss on whether the scan is of the same person (i.e. longitudinal) or not, together with (ii) a classification loss on predicting the level of vertebral bodies. The performance of this pre-trained network is then assessed on a grading classification task. We experiment on a dataset of 1016 subjects, 423 possessing follow-up scans, with the end goal of learning the disc degeneration radiological gradings attached to the intervertebral discs. We show that the performance of the pre-trained CNN on the supervised classification task is (i) superior to that of a network trained from scratch; and (ii) requires far fewer annotated training samples to reach an equivalent performance to that of the network trained from scratch.
University of Oulu（奥卢大学）；KFUPM（法赫德国王石油与矿业大学）；King Fahd Specialist Hospital Dammam（达曼法赫德国王专科医院）
Introduction: Multiple Sclerosis (MS) is a chronic disease that affects millions of people across the globe. MS can critically affect different organs of the central nervous system such as the eyes, the spinal cord, and the brain. Background: To help physicians in diagnosing MS lesions, computer-aided methods are widely used. In this regard, a considerable research has been carried out in the area of automatic detection and segmentation of MS lesions in magnetic resonance images (MRIs). Methodology: In this study, we review the different approaches that have been used in computer-aided detecti…
A weakly-supervised deep learning model for fast localisation and delineation of the skeleton, internal organs, and spinal canal on Whole-Body Diffusion-Weighted MRI (WB-DWI)
The Institute of Cancer Research（癌症研究所）；The Royal Marsden NHS Foundation Trust（皇家马斯登NHS基金会信托）；University Hospital Basel（巴塞尔大学医院）
A multi-institutional pediatric dataset of clinical radiology MRIs by the Children's Brain Tumor Network
Children’s Hospital of Philadelphia（费城儿童医院）；Perelman School of Medicine, University of Pennsylvania（宾夕法尼亚大学佩雷尔曼医学院）；National Cancer Institute（美国国家癌症研究所）；Children's National Hospital（儿童国家医院）；Sheikh Zayed Institute for Pediatric Surgical Innovation（谢赫扎耶德儿科外科创新研究院）
Personalized 4D Whole Heart Geometry Reconstruction from Cine MRI for Cardiac Digital Twins
Department of Biomedical Engineering, National University of Singapore, Singapore（新加坡国立大学生物医学工程系）；School of Data Science, Fudan University, Shanghai, China（复旦大学数据科学学院）；Department of Engineering Science, University of Oxford, Oxford, UK（牛津大学工程科学系）；Department of Medicine, National University of Singapore, Singapore（新加坡国立大学医学系）；Department of Cardiology, National University Heart Centre Singapore, Singapore（新加坡国立心脏中心心内科部）
Three-dimensional segmentation in magnetic resonance images (MRI), which reflects the true shape of the objects, is challenging since high-resolution isotropic MRIs are rare and typical MRIs are anisotropic, with the out-of-plane dimension having a much lower resolution. A potential remedy to this issue lies in the fact that often multiple sequences are acquired on different planes. However, in practice, these sequences are not orthogonal to each other, limiting the applicability of many previous solutions to reconstruct higher-resolution images from multiple lower-resolution ones. We propose …
U2AD: Uncertainty-based Unsupervised Anomaly Detection Framework for Detecting T2 Hyperintensity in MRI Spinal Cord
School of Biomedical Engineering, Shanghai Jiao Tong University（上海交通大学生物医学工程学院）；Renji Hospital, School of Medicine, Shanghai Jiao Tong University（上海交通大学医学院附属仁济医院）；Ruijin Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属瑞金医院）；National Engineering Research Center of Advanced Magnetic Resonance Technologies for Diagnosis and Therapy (NERC-AMRT)（先进磁共振诊疗技术国家工程研究中心）；Med-X Research Institute, Shanghai Jiao Tong University（上海交通大学Med-X研究院）
Accurate segmentation of brain resection cavities (RCs) aids in postoperative analysis and determining follow-up treatment. Convolutional neural networks (CNNs) are the state-of-the-art image segmentation technique, but require large annotated datasets for training. Annotation of 3D medical images is time-consuming, requires highly-trained raters, and may suffer from high inter-rater variability. Self-supervised learning strategies can leverage unlabeled data for training. We developed an algorithm to simulate resections from preoperative magnetic resonance images (MRIs). We performed self-sup…
The Hong Kong University of Science and Technology（香港科技大学）；Huazhong University of Science and Technology（华中科技大学）
Medical anomaly detection is a crucial yet challenging task aimed at recognizing abnormal images to assist in diagnosis. Due to the high-cost annotations of abnormal images, most methods utilize only known normal images during training and identify samples deviating from the normal profile as anomalies in the testing phase. Many readily available unlabeled images containing anomalies are thus ignored in the training phase, restricting the performance. To solve this problem, we introduce one-class semi-supervised learning (OC-SSL) to utilize known normal and unlabeled images for training, and p…
DyABD: The Abdominal Muscle Segmentation in Dynamic MRI Benchmark
Science Foundation Ireland Centre for Research Training in Machine Learning（科学基金会爱尔兰机器学习研究培训中心）；Aix Marseille Univ, Univ Gustave Eiffel, LBA, Marseille France（马赛大学，奥古斯特·埃菲尔大学，LBA，法国马赛）；Aix Marseille Université, CNRS, CRMBM UMR 7339, Marseille France（马赛大学，法国国家科学研究中心，CRMBM UMR 7339，法国马赛）；Insight Centre for Data Analytics, University College Dublin（都柏林大学数据分析洞察中心）
Quantitative mapping from conventional MRI using self-supervised physics-guided deep learning: applications to a large-scale, clinically heterogeneous dataset
Automatic segmentation of brain abnormalities is challenging, as they vary considerably from one pathology to another. Current methods are supervised and require numerous annotated images for each pathology, a strenuous task. To tackle anatomical variability, Unsupervised Anomaly Detection (UAD) methods are proposed, detecting anomalies as outliers of a healthy model learned using a Variational Autoencoder (VAE). Previous work on UAD adopted a 2D approach, meaning that MRIs are processed as a collection of independent slices. Yet, it does not fully exploit the spatial information contained in …
We propose a novel framework for Alzheimer's disease (AD) detection using brain MRIs. The framework starts with a data augmentation method called Brain-Aware Replacements (BAR), which leverages a standard brain parcellation to replace medically-relevant 3D brain regions in an anchor MRI from a randomly picked MRI to create synthetic samples. Ground truth "hard" labels are also linearly mixed depending on the replacement ratio in order to create "soft" labels. BAR produces a great variety of realistic-looking synthetic MRIs with higher local variability compared to other mix-based methods, such…
Technische Universität München（慕尼黑工业大学）；ScintHealth GmbH（ScintHealth有限公司）；Reply SpA（Reply股份公司）；Helmholtz Zentrum München（慕尼黑亥姆霍兹中心）；Helmholtz AI（亥姆霍兹人工智能研究所）
CT and MRI are two of the most informative modalities in spinal diagnostics and treatment planning. CT is useful when analysing bony structures, while MRI gives information about the soft tissue. Thus, fusing the information of both modalities can be very beneficial. Registration is the first step for this fusion. While the soft tissues around the vertebra are deformable, each vertebral body is constrained to move rigidly. We propose a weakly-supervised deep learning framework that preserves the rigidity and the volume of each vertebra while maximizing the accuracy of the registration. To achi…
Longitudinal MRIs are often used to capture the gradual deterioration of brain structure and function caused by aging or neurological diseases. Analyzing this data via machine learning generally requires a large number of ground-truth labels, which are often missing or expensive to obtain. Reducing the need for labels, we propose a self-supervised strategy for representation learning named Longitudinal Neighborhood Embedding (LNE). Motivated by concepts in contrastive learning, LNE explicitly models the similarity between trajectory vectors across different subjects. We do so by building a gra…
Optimization in Sparse 2D to Dense 3D Weakly Supervised Learning: Application to Multi-Label Segmentation of Large ex vivo MRI Data
NeuroPoly Lab, Institute of Biomedical Engineering, Polytechnique Montreal（神经多极实验室，生物医学工程学院，蒙特利尔理工学院）；École Centrale de Lyon（里昂中央理工学院）；Mila - Quebec AI Institute（魁北克人工智能研究所）；Functional Neuroimaging Unit, CRIUGM, University of Montreal（功能神经影像单元，CRIUGM，蒙特利尔大学）；Translational Neuroradiology Section, National Institute of Neurological Disorders and Stroke, National Institutes of Health（转化神经放射学部门，国家神经疾病与中风研究所，国家卫生研究院）；Translational Imaging in Neurology (ThINk) Basel, Department of Biomedical Engineering, Faculty of Medicine, University Hospital Basel and University of Basel（神经学转化成像（ThINk）巴塞尔，生物医学工程系，医学院，巴塞尔大学医院和巴塞尔大学）；Neurologic Clinic and Policlinic, Departments of Medicine, University Hospital Basel, Switzerland（神经科诊所和多科诊所，医学院，巴塞尔大学医院，瑞士）；Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB), University Hospital Basel and University of Basel, Switzerland（临床神经免疫学和神经科学巴塞尔研究中心（RC2NB），巴塞尔大学医院和巴塞尔大学，瑞士）；National Institute of Neurological Disorders and Stroke, National Institutes of Health（国家神经疾病与中风研究所，国家卫生研究院）；Centre de recherche du CHU Sainte-Justine, Université de Montréal, Montreal, QC, Canada（圣朱斯特医院研究中心，蒙特利尔大学，蒙特利尔，魁北克，加拿大）；Quantitative MRI core facility, NINDS, NIH（定量MRI核心设施，NINDS，NIH）；Experimental Immunotherapeutics Unit, Division of Neuroimmunology and Neurovirology, NINDS, NIH（实验免疫治疗单元，神经免疫学和神经病毒学部门，NINDS，NIH）
National University of Singapore（新加坡国立大学）；National University Hospital（国立大学医院）；Zhejiang University（浙江大学）
In this paper, we propose a novel semi-supervised learning (SSL) framework named BoostMIS that combines adaptive pseudo labeling and informative active annotation to unleash the potential of medical image SSL models: (1) BoostMIS can adaptively leverage the cluster assumption and consistency regularization of the unlabeled data according to the current learning status. This strategy can adaptively generate one-hot "hard" labels converted from task model predictions for better task model training. (2) For the unselected unlabeled images with low confidence, we introduce an Active learning (AL) …
King’s College London（伦敦国王学院）；University Hospitals Leuven（鲁汶大学医院）；Technical University of Munich（慕尼黑工业大学）；University College London（伦敦大学学院）
This paper describes our method for our participation in the FeTA challenge2021 (team name: TRABIT). The performance of convolutional neural networks for medical image segmentation is thought to correlate positively with the number of training data. The FeTA challenge does not restrict participants to using only the provided training data but also allows for using other publicly available sources. Yet, open access fetal brain data remains limited. An advantageous strategy could thus be to expand the training data to cover broader perinatal brain imaging sources. Perinatal brain MRIs, other tha…
Deep cerebellar nuclei are a key structure of the cerebellum that are involved in processing motor and sensory information. It is thus a crucial step to accurately segment deep cerebellar nuclei for the understanding of the cerebellum system and its utility in deep brain stimulation treatment. However, it is challenging to clearly visualize such small nuclei under standard clinical magnetic resonance imaging (MRI) protocols and therefore precise segmentation is not feasible. Recent advances in 7 Tesla (T) MRI technology and great potential of deep neural networks facilitate automatic patient-s…
The recent achievements of Deep Learning rely on the test data being similar in distribution to the training data. In an ideal case, Deep Learning models would achieve Out-of-Distribution (OoD) Generalization, i.e. reliably make predictions on out-of-distribution data. Yet in practice, models usually fail to generalize well when facing a shift in distribution. Several methods were thereby designed to improve the robustness of the features learned by a model through Regularization- or Domain-Prediction-based schemes. Segmenting medical images such as MRIs of the hippocampus is essential for the…
Automatic segmentation of vertebral bodies (VBs) and intervertebral discs (IVDs) in 3D magnetic resonance (MR) images is vital in diagnosing and treating spinal diseases. However, segmenting the VBs and IVDs simultaneously is not trivial. Moreover, problems exist, including blurry segmentation caused by anisotropy resolution, high computational cost, inter-class similarity and intra-class variability, and data imbalances. We proposed a two-stage algorithm, named semi-supervised hybrid spine network (SSHSNet), to address these problems by achieving accurate simultaneous VB and IVD segmentation.…
Machine learning analysis of longitudinal neuroimaging data is typically based on supervised learning, which requires a large number of ground-truth labels to be informative. As ground-truth labels are often missing or expensive to obtain in neuroscience, we avoid them in our analysis by combing factor disentanglement with self-supervised learning to identify changes and consistencies across the multiple MRIs acquired of each individual over time. Specifically, we propose a new definition of disentanglement by formulating a multivariate mapping between factors (e.g., brain age) associated with…
Medical University of Vienna（维也纳医科大学）；Intel（英特尔公司）；Lobachevsky State University（罗巴切夫斯基国立大学）
The segmentation of brain tumors in multimodal MRIs is one of the most challenging tasks in medical image analysis. The recent state of the art algorithms solving this task is based on machine learning approaches and deep learning in particular. The amount of data used for training such models and its variability is a keystone for building an algorithm with high representation power. In this paper, we study the relationship between the performance of the model and the amount of data employed during the training process. On the example of brain tumor segmentation challenge, we compare the model…
Training-free image style alignment for self-adapting domain shift on handheld ultrasound devices
School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）；Shanghai Advanced Research Institute, Chinese Academy of Sciences（中国科学院上海高等研究院）；University of Chinese Academy of Sciences（中国科学院大学）；College of Computer Science, Sichuan University（四川大学计算机学院）；College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）；The Chinese University of Hong Kong（香港中文大学）；Beth Israel Deaconess Medical Center, Harvard Medical School（哈佛医学院贝斯以色列女执事医疗中心）；Institute of High Performance Computing, Agency for Science, Technology and Research（新加坡科技研究局高性能计算研究所）
