Few-shot brain segmentation from weakly labeled data with deep heteroscedastic multi-task networks
Support Centre for Advanced Neuroimaging, University Institute of Diagnostic and Interventional Neuroradiology, Inselspital, Bern University Hospital, Bern, Switzerland（伯尔尼大学医院 Inselspital 诊断与介入神经放射学大学研究所高级神经影像支持中心（伯尔尼））；Insel Data Science Centre, Inselspital, Bern University Hospital, University of Bern, Switzerland（伯尔尼大学医院 Inselspital Insel 数据科学中心（伯尔尼））；University of Bern, Switzerland（伯尔尼大学）
Richard McKinley, Michael Rebsamen, Raphael Meier, Mauricio Reyes, Christian Rummel, Roland Wiest
In applications of supervised learning applied to medical image segmentation, the need for large amounts of labeled data typically goes unquestioned. In particular, in the case of brain anatomy segmentation, hundreds or thousands of weakly-labeled volumes are often used as training data. In this paper, we first observe that for many brain structures, a small number of training examples, (n=9), weakly labeled using Freesurfer 6.0, plus simple data augmentation, suffice as training data to achieve high performance, achieving an overall mean Dice coefficient of $0.84 \pm 0.12$ compared to Freesurfer over 28 brain structures in T1-weighted images of $\approx 4000$ 9-10 year-olds from the Adolescent Brain Cognitive Development study. We then examine two varieties of heteroscedastic network as a method for improving classification results. An existing proposal by Kendall and Gal, which uses Monte-Carlo inference to learn to predict the variance of each prediction, yields an overall mean Dice of $0.85 \pm 0.14$ and showed statistically significant improvements over 25 brain structures. Meanwhile a novel heteroscedastic network which directly learns the probability that an example has been mislabeled yielded an overall mean Dice of $0.87 \pm 0.11$ and showed statistically significant improvements over all but one of the brain structures considered. The loss function associated to this network can be interpreted as performing a form of learned label smoothing, where labels are only smoothed where they are judged to be uncertain.
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Central South University（中南大学）；Shenzhen University（深圳大学）
The state-of-the-art deep learning methods have demonstrated impressive performance in segmentation tasks. However, the success of these methods depends on a large amount of manually labeled masks, which are expensive and time-consuming to be collected. In this work, a novel Consistent PerceptionGenerative Adversarial Network (CPGAN) is proposed for semi-supervised stroke lesion segmentation. The proposed CPGAN can reduce the reliance on fully labeled samples. Specifically, A similarity connection module (SCM) is designed to capture the information of multi-scale features. The proposed SCM can…
In the past few years, deep learning (DL) models have drawn great attention and shown superior performance on brain tumor and subregion segmentation tasks. However, the success is limited to segmentation of adult gliomas, where sufficient data have been collected, manually labeled, and published for training DL models. It is still challenging to segment pediatric tumors, because the appearances are different from adult gliomas. Hence, directly applying a pretained DL model on pediatric data usually generates unacceptable results. Because pediatric data is very limited, both labeled and unlabel…
Charité Universitätsmedizin Berlin（柏林夏里特医学院）；Technical University Berlin（柏林工业大学）；Birmingham City University（伯明翰城市大学）；Fraunhofer MEVIS（弗劳恩霍夫医学图像计算研究所）；Beuth University of Applied Sciences（拜伊特应用科学大学）
Anonymization and data sharing are crucial for privacy protection and acquisition of large datasets for medical image analysis. This is a big challenge, especially for neuroimaging. Here, the brain's unique structure allows for re-identification and thus requires non-conventional anonymization. Generative adversarial networks (GANs) have the potential to provide anonymous images while preserving predictive properties. Analyzing brain vessel segmentation, we trained 3 GANs on time-of-flight (TOF) magnetic resonance angiography (MRA) patches for image-label generation: 1) Deep convolutional GAN,…
Structure Matters: Brain Graph Augmentation via Learnable Edge Masking for Data-efficient Psychiatric Diagnosis
Institute of Innovation, Science and Sustainability（创新、科学与可持续发展研究所）；Federation University Australia（联邦大学澳大利亚分校）；School of Information and Electronic Engineering（信息与电子工程学院）；Zhejiang Gongshang University（浙江工商大学）；School of Computing Technologies（计算技术学院）；RMIT University（皇家墨尔本理工学院）；School of Computing and Information Systems（计算与信息系统学院）；The University of Melbourne（墨尔本大学）
Medical University of Vienna（维也纳医科大学）；Intel（英特尔公司）；Lobachevsky State University（罗巴切夫斯基国立大学）
The segmentation of brain tumors in multimodal MRIs is one of the most challenging tasks in medical image analysis. The recent state of the art algorithms solving this task is based on machine learning approaches and deep learning in particular. The amount of data used for training such models and its variability is a keystone for building an algorithm with high representation power. In this paper, we study the relationship between the performance of the model and the amount of data employed during the training process. On the example of brain tumor segmentation challenge, we compare the model…
University of Copenhagen（哥本哈根大学）；Cerebriu A/S（Cerebriu公司）
Medical images used in clinical practice are heterogeneous and not the same quality as scans studied in academic research. Preprocessing breaks down in extreme cases when anatomy, artifacts, or imaging parameters are unusual or protocols are different. Methods robust to these variations are most needed. A novel deep learning method is proposed for fast and accurate segmentation of the human brain into 132 regions. The proposed model uses an efficient U-Net-like network and benefits from the intersection points of different views and hierarchical relations for the fusion of the orthogonal 2D pl…
University at Buffalo, The State University of New York（纽约州立大学布法罗分校）
The recent state-of-the-art deep learning methods have significantly improved brain tumor segmentation. However, fully supervised training requires a large amount of manually labeled masks, which is highly time-consuming and needs domain expertise. Weakly supervised learning with scribbles provides a good trade-off between model accuracy and the effort of manual labeling. However, for segmenting the hierarchical brain tumor structures, manually labeling scribbles for each substructure could still be demanding. In this paper, we use only two kinds of weak labels, i.e., scribbles on whole tumor …
SingleStrip: learning skull-stripping from a single labeled example
Department of Medical Imaging Sciences, University of Haifa, Israel（海法大学医学影像科学系）；Athinoula A.~Martinos Center for Biomedical Imaging, Boston, MA, USA（Athinoula A.马丁诺斯生物医学影像中心）；Department of Radiology, Harvard Medical School, Boston, MA, USA（哈佛医学院放射科）；Department of Radiology, Massachusetts General Hospital, Boston, MA, USA（麻省总医院放射科）
King’s College London（伦敦国王学院）；Biomediq A/S（Biomediq公司）；University College London（伦敦大学学院）；Chelsea and Westminster Hospital NHS Foundation Trust（切尔西和威斯敏斯特医院NHS基金会信托）；Cereriu A/S（Cereriu公司）
Supervised learning algorithms trained on medical images will often fail to generalize across changes in acquisition parameters. Recent work in domain adaptation addresses this challenge and successfully leverages labeled data in a source domain to perform well on an unlabeled target domain. Inspired by recent work in semi-supervised learning we introduce a novel method to adapt from one source domain to $n$ target domains (as long as there is paired data covering all domains). Our multi-domain adaptation method utilises a consistency loss combined with adversarial learning. We provide results…
Over the last few years, deep learning has proven to be a great solution to many problems, such as image or text classification. Recently, deep learning-based solutions have outperformed humans on selected benchmark datasets, yielding a promising future for scientific and real-world applications. Training of deep learning models requires vast amounts of high quality data to achieve such supreme performance. In real-world scenarios, obtaining a large, coherent, and properly labeled dataset is a challenging task. This is especially true in medical applications, where high-quality data and annota…
Novosibirsk State University（新西伯利亚国立大学）；International Tomography Center SB RAS（俄罗斯科学院西伯利亚分院国际断层成像中心）；Sobolev Institute of Mathematics SB RAS（俄罗斯科学院西伯利亚分院索博列夫数学研究所）
This paper presents an automatic algorithm for the segmentation of areas affected by an acute stroke on the non-contrast computed tomography brain images. The proposed algorithm is designed for learning in a weakly supervised scenario when some images are labeled accurately, and some images are labeled inaccurately. Wrong labels appear as a result of inaccuracy made by a radiologist in the process of manual annotation of computed tomography images. We propose methods for solving the segmentation problem in the case of inaccurately labeled training data. We use the U-Net neural network architec…
University of Maryland School of Medicine（马里兰大学医学院）
Purpose: Arterial spin labeling (ASL) perfusion imaging indicates direct and absolute measurement of cerebral blood flow (CBF). Arterial transit time (ATT) is a related physiological parameter reflecting the duration for the labeled spins to reach the brain region of interest. Multiple post-labeling delay (PLDs) can provide robust measures of both CBF and ATT, allowing for optimization of regional CBF modeling based on ATT. The prolonged acquisition time can potentially reduce the quality and accuracy of the CBF and ATT estimation. We proposed a novel network to significantly reduce the number…
University of Copenhagen（哥本哈根大学）；Erasmus Medical Center（伊拉斯谟医学中心）
Suppose one is faced with the challenge of tissue segmentation in MR images, without annotators at their center to provide labeled training data. One option is to go to another medical center for a trained classifier. Sadly, tissue classifiers do not generalize well across centers due to voxel intensity shifts caused by center-specific acquisition protocols. However, certain aspects of segmentations, such as spatial smoothness, remain relatively consistent and can be learned separately. Here we present a smoothness prior that is fit to segmentations produced at another medical center. This inf…
Universidad de Buenos Aires（布宜诺斯艾利斯大学）；CONICET（国家科学技术研究委员会）；Universidad Nacional del Litoral（滨海国立大学）
Brain lesion and anatomy segmentation in magnetic resonance images are fundamental tasks in neuroimaging research and clinical practice. Given enough training data, convolutional neuronal networks (CNN) proved to outperform all existent techniques in both tasks independently. However, to date, little work has been done regarding simultaneous learning of brain lesion and anatomy segmentation from disjoint datasets. In this work we focus on training a single CNN model to predict brain tissue and lesion segmentations using heterogeneous datasets labeled independently, according to only one of the…
Learn2Synth: Learning Optimal Data Synthesis Using Hypergradients for Brain Image Segmentation
Massachusetts General Hospital and Harvard Medical School（麻省总医院和哈佛医学院）；Danish Research Centre for Magnetic Resonance, Copenhagen University Hospital（丹麦磁共振研究中心，哥本哈根大学医院）；Centre for Medical Image Computing, University College London（医学影像计算中心，伦敦大学学院）；Computer Science and AI Laboratory, Massachusetts Institute of Technology（计算机科学与人工智能实验室，麻省理工学院）；Department of Experimental Psychology, University College London（实验心理学系，伦敦大学学院）
Advances in computational cognitive neuroimaging research are related to the availability of large amounts of labeled brain imaging data, but such data are scarce and expensive to generate. While powerful data generation mechanisms, such as Generative Adversarial Networks (GANs), have been designed in the last decade for computer vision, such improvements have not yet carried over to brain imaging. A likely reason is that GANs training is ill-suited to the noisy, high-dimensional and small-sample data available in functional neuroimaging. In this paper, we introduce Conditional Independent Com…
Biomedical text tagging systems are plagued by the dearth of labeled training data. There have been recent attempts at using pre-trained encoders to deal with this issue. Pre-trained encoder provides representation of the input text which is then fed to task-specific layers for classification. The entire network is fine-tuned on the labeled data from the target task. Unfortunately, a low-resource biomedical task often has too few labeled instances for satisfactory fine-tuning. Also, if the label space is large, it contains few or no labeled instances for majority of the labels. Most biomedical…
Few-Shot Continual Learning for 3D Brain MRI with Frozen Foundation Models
Independent Researcher, USA（美国独立研究者）；Indiana University at Bloomington, United States（印第安纳大学布卢明顿分校）；Independent Researcher, Taiwan（台湾独立研究者）；National Taiwan University, Taiwan（台湾国立大学）；Boise State University, United States（博伊西州立大学）；National Yang Ming Chiao Tung University, Taiwan（台湾阳明交通大学）
Epilepsy is a chronic neurological disorder affecting more than 65 million people worldwide and manifested by recurrent unprovoked seizures. The unpredictability of seizures not only degrades the quality of life of the patients, but it can also be life-threatening. Modern systems monitoring electroencephalography (EEG) signals are being currently developed with the view to detect epileptic seizures in order to alert caregivers and reduce the impact of seizures on patients' quality of life. Such seizure detection systems employ state-of-the-art machine learning algorithms that require a conside…
Training a deep neural network is an optimization problem with four main ingredients: the design of the deep neural network, the per-sample loss function, the population loss function, and the optimizer. However, methods developed to compete in recent BraTS challenges tend to focus only on the design of deep neural network architectures, while paying less attention to the three other aspects. In this paper, we experimented with adopting the opposite approach. We stuck to a generic and state-of-the-art 3D U-Net architecture and experimented with a non-standard per-sample loss function, the gene…
The Weizmann Institute of Science（魏茨曼科学研究所）；Zuckerman Institute（朱克曼研究所）；Columbia University（哥伦比亚大学）
Reconstructing observed images from fMRI brain recordings is challenging. Unfortunately, acquiring sufficient "labeled" pairs of {Image, fMRI} (i.e., images with their corresponding fMRI responses) to span the huge space of natural images is prohibitive for many reasons. We present a novel approach which, in addition to the scarce labeled data (training pairs), allows to train fMRI-to-image reconstruction networks also on "unlabeled" data (i.e., images without fMRI recording, and fMRI recording without images). The proposed model utilizes both an Encoder network (image-to-fMRI) and a Decoder n…
Semi-supervised learning has the potential to improve the data-efficiency of training data-hungry deep neural networks, which is especially important for medical image analysis tasks where labeled data is scarce. In this work, we present a simple semi-supervised learning method for lesion segmentation tasks based on the ideas of cut-paste augmentation and consistency regularization. By exploiting the mask information available in the labeled data, we synthesize partially labeled samples from the unlabeled images so that the usual supervised learning objective (e.g., binary cross entropy) can b…
University of Technology Sydney（悉尼科技大学）；Cuvos Pty. Ltd.（Cuvos私人有限公司）；Queensland University of Technology（昆士兰科技大学）
The surge in interest in Artificial Intelligence (AI) over the past decade has been driven almost exclusively by advances in Artificial Neural Networks (ANNs). While ANNs set state-of-the-art performance for many previously intractable problems, the use of global gradient descent necessitates large datasets and computational resources for training, potentially limiting their scalability for real-world domains. Spiking Neural Networks (SNNs) are an alternative to ANNs that use more brain-like artificial neurons and can use local unsupervised learning to rapidly discover sparse recognizable feat…
The Academic College of Tel Aviv-Yaffo（特拉维夫-雅法学术学院）；Dartmouth College（达特茅斯学院）；The Edmond and Lily Safra Center for Brain Sciences at The Hebrew University of Jerusalem（耶路撒冷希伯来大学埃德蒙和莉莉·萨夫拉脑科学中心）
A Privacy-Preserving Domain Adversarial Federated learning for multi-site brain functional connectivity analysis
School of Energy and Electrical Engineering, Chang'an University（长安大学能源与电气工程学院）；Tulane University（杜兰大学）；School of Data Science, University of Virginia（弗吉尼亚大学数据科学学院）；Georgia State University（佐治亚州立大学）；Georgia Institute of Technology（佐治亚理工学院）；Emory University（埃默里大学）
University of Cologne（科隆大学）；University of Washington（华盛顿大学）
Making an informed, correct and quick decision can be life-saving. It's crucial for animals during an escape behaviour or for autonomous cars during driving. The decision can be complex and may involve an assessment of the amount of threats present and the nature of each threat. Thus, we should expect early sensory processing to supply classification information fast and accurately, even before relying the information to higher brain areas or more complex system components downstream. Today, advanced convolutional artificial neural networks can successfully solve visual detection and classific…
Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）；University of Sydney（悉尼大学）；University of New South Wales（新南威尔士大学）；Massachusetts General Hospital（麻省总医院）
Diffusion MRI tractography is an advanced imaging technique that enables in vivo mapping of the brain's white matter connections. White matter parcellation classifies tractography streamlines into clusters or anatomically meaningful tracts. It enables quantification and visualization of whole-brain tractography. Currently, most parcellation methods focus on the deep white matter (DWM), whereas fewer methods address the superficial white matter (SWM) due to its complexity. We propose a novel two-stage deep-learning-based framework, Superficial White Matter Analysis (SupWMA), that performs an ef…
DataNova；U.S. Army Research Laboratory（美国陆军研究实验室）；University of Texas at San Antonio（德克萨斯大学圣安东尼奥分校）
Electroencephalography (EEG) headsets are the most commonly used sensing devices for Brain-Computer Interface. In real-world applications, there are advantages to extrapolating data from one user session to another. However, these advantages are limited if the data arise from different hardware systems, which often vary between application spaces. Currently, this creates a need to recalibrate classifiers, which negatively affects people's interest in using such systems. In this paper, we employ active weighted adaptation regularization (AwAR), which integrates weighted adaptation regularizatio…
Harvard University（哈佛大学）；MIT-IBM Watson AI Lab（麻省理工-IBM沃森人工智能实验室）；IBM Research（IBM研究院）
While deep neural networks take loose inspiration from neuroscience, it is an open question how seriously to take the analogies between artificial deep networks and biological neuronal systems. Interestingly, recent work has shown that deep convolutional neural networks (CNNs) trained on large-scale image recognition tasks can serve as strikingly good models for predicting the responses of neurons in visual cortex to visual stimuli, suggesting that analogies between artificial and biological neural networks may be more than superficial. However, while CNNs capture key properties of the average…
