Deep learning has proven to be an essential tool for medical image analysis. However, the need for accurately labeled input data, often requiring time- and labor-intensive annotation by experts, is a major limitation to the use of deep learning. One solution to this challenge is to allow for use of coarse or noisy labels, which could permit more efficient and scalable labeling of images. In this work, we develop a lopsided loss function based on entropy regularization that assumes the existence of a nontrivial false negative rate in the target annotations. Starting with a carefully annotated brain metastasis lesion dataset, we simulate data with false negatives by (1) randomly censoring the annotated lesions and (2) systematically censoring the smallest lesions. The latter better models true physician error because smaller lesions are harder to notice than the larger ones. Even with a simulated false negative rate as high as 50%, applying our loss function to randomly censored data preserves maximum sensitivity at 97% of the baseline with uncensored training data, compared to just 10% for a standard loss function. For the size-based censorship, performance is restored from 17% with the current standard to 88% with our lopsided bootstrap loss. Our work will enable more efficient scaling of the image labeling process, in parallel with other approaches on creating more efficient user interfaces and tools for annotation.
Universitätsklinikum Erlangen（埃尔朗根大学医院）；Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)（埃尔朗根-纽伦堡弗里德里希-亚历山大大学（FAU））；Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN)（埃尔朗根-EMN综合癌症中心（CCC ER-EMN））
Brain metastases occur frequently in patients with metastatic cancer. Early and accurate detection of brain metastases is very essential for treatment planning and prognosis in radiation therapy. To improve brain metastasis detection performance with deep learning, a custom detection loss called volume-level sensitivity-specificity (VSS) is proposed, which rates individual metastasis detection sensitivity and specificity in (sub-)volume levels. As sensitivity and precision are always a trade-off in a metastasis level, either a high sensitivity or a high precision can be achieved by adjusting t…
Stanford University（斯坦福大学）；Oslo University Hospital（奥斯陆大学医院）
Magnetic resonance (MR) imaging is an essential diagnostic tool in clinical medicine. Recently, a variety of deep learning methods have been applied to segmentation tasks in medical images, with promising results for computer-aided diagnosis. For MR images, effectively integrating different pulse sequences is important to optimize performance. However, the best way to integrate different pulse sequences remains unclear. In this study, we evaluate multiple architectural features and characterize their effects in the task of metastasis segmentation. Specifically, we consider (1) different pulse …
Memorial Sloan Kettering Cancer Center（纪念斯隆·凯特琳癌症中心）；Indian Institute of Technology Madras（印度理工学院马德拉斯分校）
Artificial intelligence (AI) in radiology has made great strides in recent years, but many hurdles remain. Overfitting and lack of generalizability represent important ongoing challenges hindering accurate and dependable clinical deployment. If AI algorithms can avoid overfitting and achieve true generalizability, they can go from the research realm to the forefront of clinical work. Recently, small data AI approaches such as deep neuroevolution (DNE) have avoided overfitting small training sets. We seek to address both overfitting and generalizability by applying DNE to a virtually pooled dat…
Large-Scale Pre-training Enables Multimodal AI Differentiation of Radiation Necrosis from Brain Metastasis Progression on Routine MRI
University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根大学医院，埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN)（埃尔朗根-欧洲大都会区纽伦堡综合癌症中心）；Bavarian Cancer Research Center (BZKF)（巴伐利亚癌症研究中心）；Universität Würzburg（维尔茨堡大学）；Peking University（北京大学）；Universitätsklinikum Würzburg（维尔茨堡大学医院）
University Hospital Erlangen（埃尔朗根大学医院）；Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN)（埃尔朗根-EMN综合癌症中心）；Pattern Recognition Lab（模式识别实验室）
Stereotactic radiotherapy (SRT) is one of the most important treatment for patients with brain metastases (BM). Conventionally, following SRT patients are monitored by serial imaging and receive salvage treatments in case of significant tumor growth. We hypothesized that using radiomics and machine learning (ML), metastases at high risk for subsequent progression could be identified during follow-up prior to the onset of significant tumor growth, enabling personalized follow-up intervals and early selection for salvage treatment. All experiments are performed on a dataset from clinical routine…
The detection of brain metastases (BM) in their early stages could have a positive impact on the outcome of cancer patients. We previously developed a framework for detecting small BM (with diameters of less than 15mm) in T1-weighted Contrast-Enhanced 3D Magnetic Resonance images (T1c) to assist medical experts in this time-sensitive and high-stakes task. The framework utilizes a dedicated convolutional neural network (CNN) trained using labeled T1c data, where the ground truth BM segmentations were provided by a radiologist. This study aims to advance the framework with a noisy student-based …
Google Brain（谷歌大脑）；Google Inc（谷歌公司）；Verily Life Sciences（Verily生命科学）
Each year, the treatment decisions for more than 230,000 breast cancer patients in the U.S. hinge on whether the cancer has metastasized away from the breast. Metastasis detection is currently performed by pathologists reviewing large expanses of biological tissues. This process is labor intensive and error-prone. We present a framework to automatically detect and localize tumors as small as 100 x 100 pixels in gigapixel microscopy images sized 100,000 x 100,000 pixels. Our method leverages a convolutional neural network (CNN) architecture and obtains state-of-the-art results on the Camelyon16…
The Diagnosis a Reporter Leaves Unspoken: Surfacing Frozen Tumor Features for Brain-Tumor MRI Reporting
National University of Sciences and Technology（国立科技大学）；Consultant Radiologist（顾问放射医师）；Bahria University Health Sciences Campus（巴里亚大学健康科学校区）；Faculty of Science, Ontario Tech University（安大略理工大学理学院）
Southern Methodist University（南卫理公会大学）；University of Texas at Dallas（德克萨斯大学达拉斯分校）；University of Texas Southwestern Medical Center（德克萨斯大学西南医学中心）
Tumor shape is a key factor that affects tumor growth and metastasis. This paper proposes a topological feature computed by persistent homology to characterize tumor progression from digital pathology and radiology images and examines its effect on the time-to-event data. The proposed topological features are invariant to scale-preserving transformation and can summarize various tumor shape patterns. The topological features are represented in functional space and used as functional predictors in a functional Cox proportional hazards model. The proposed model enables interpretable inference ab…
The Institute of Medical Robotics, Shanghai Jiao Tong University（上海交通大学医疗机器人研究所）；The Hamlyn Centre for Robotic Surgery, Imperial College London（帝国理工学院哈姆林机器人手术中心）；Queen Mary University of London（伦敦玛丽女王大学）；Hangzhou Dianzi University（杭州电子科技大学）
Segmentation of brain tumors and their subregions remains a challenging task due to their weak features and deformable shapes. In this paper, three patterns (cross-skip, skip-1 and skip-2) of distributed dense connections (DDCs) are proposed to enhance feature reuse and propagation of CNNs by constructing tunnels between key layers of the network. For better detecting and segmenting brain tumors from multi-modal 3D MR images, CNN-based models embedded with DDCs (DDU-Nets) are trained efficiently from pixel to pixel with a limited number of parameters. Postprocessing is then applied to refine t…
BrainNormalizer: Anatomy-Informed Pseudo-Healthy Brain Reconstruction from Tumor MRI via Edge-Guided ControlNet
University of Pittsburgh（匹兹堡大学）；Georgia Institute of Technology（佐治亚理工学院）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；Cedars-Sinai Medical Center（西德萨斯医疗中心）；Mayo Clinic Arizona（梅奥诊所亚利桑那分部）
Memorial Sloan Kettering Cancer Center（纪念斯隆-凯特琳癌症中心）；Indian Institute of Technology, Madras（马德拉斯印度理工学院）
Purpose: Image classification is perhaps the most fundamental task in imaging AI. However, labeling images is time-consuming and tedious. We have recently demonstrated that reinforcement learning (RL) can classify 2D slices of MRI brain images with high accuracy. Here we make two important steps toward speeding image classification: Firstly, we automatically extract class labels from the clinical reports. Secondly, we extend our prior 2D classification work to fully 3D image volumes from our institution. Hence, we proceed as follows: in Part 1, we extract labels from reports automatically usin…
Oslo University Hospital（奥斯陆大学医院）；Stanford University（斯坦福大学）
The purpose was to assess the clinical value of a novel DropOut model for detecting and segmenting brain metastases, in which a neural network is trained on four distinct MRI sequences using an input dropout layer, thus simulating the scenario of missing MRI data by training on the full set and all possible subsets of the input data. This retrospective, multi-center study, evaluated 165 patients with brain metastases. A deep learning based segmentation model for automatic segmentation of brain metastases, named DropOut, was trained on multi-sequence MRI from 100 patients, and validated/tested …
Automatic detection of brain neoplasm in Magnetic Resonance Imaging (MRI) is gaining importance in many medical diagnostic applications. This report presents two improvements for brain neoplasm detection in MRI data: an advanced preprocessing technique is proposed to improve the area of interest in MRI data and a hybrid technique using Convolutional Neural Network (CNN) for feature extraction followed by Support Vector Machine (SVM) for classification. The learning algorithm for SVM is modified with the addition of cost function to minimize false positive prediction addressing the errors in MR…
Interpretable Motion Artificat Detection in structural Brain MRI
Indian Institute of Technology, Madras（印度理工学院，马德拉斯）；Boston Childrens Hospital, Harvard Medical School（波士顿儿童医院，哈佛医学院）；Indian Institute of Technology, Bhilai（印度理工学院，比哈尔）
Evaluating the Safety of Deep Learning-Based Brain MRI Reconstruction
School of Science, Engineering & Technology, RMIT University Vietnam（RMIT越南大学科学、工程与技术学院）；School of Computing Technologies, RMIT University（RMIT大学计算技术学院）；School of Health and Biomedical Science, RMIT University（RMIT大学健康与生物医学科学学院）
Indian Statistical Institute（印度统计研究所）；DAV Institute of Engineering and Technology（DAV工程技术学院）
Inspired by the success of Convolutional Neural Networks (CNN), we develop a novel Computer Aided Detection (CADe) system using CNN for Glioblastoma Multiforme (GBM) detection and segmentation from multi channel MRI data. A two-stage approach first identifies the presence of GBM. This is followed by a GBM localization in each "abnormal" MR slice. As part of the CADe system, two CNN architectures viz. Classification CNN (C-CNN) and Detection CNN (D-CNN) are employed. The CADe system considers MRI data consisting of four sequences ($T_1$, $T_{1c},$ $T_2$, and $T_{2FLAIR}$) as input, and automati…
Hamburg University of Technology（汉堡工业大学）；Jung Diagnostics GmbH（荣格诊断有限公司）
The detection of lesions in magnetic resonance imaging (MRI)-scans of human brains remains challenging, time-consuming and error-prone. Recently, unsupervised anomaly detection (UAD) methods have shown promising results for this task. These methods rely on training data sets that solely contain healthy samples. Compared to supervised approaches, this significantly reduces the need for an extensive amount of labeled training data. However, data labelling remains error-prone. We study how unhealthy samples within the training data affect anomaly detection performance for brain MRI-scans. For our…
Henry M. Jackson Foundation（亨利·M·杰克逊基金会）；National Institute of Health（美国国立卫生研究院）；Johns Hopkins University（约翰斯·霍普金斯大学）
Multiple Sclerosis (MS) is an autoimmune disease that leads to lesions in the central nervous system. Magnetic resonance (MR) images provide sufficient imaging contrast to visualize and detect lesions, particularly those in the white matter. Quantitative measures based on various features of lesions have been shown to be useful in clinical trials for evaluating therapies. Therefore robust and accurate segmentation of white matter lesions from MR images can provide important information about the disease status and progression. In this paper, we propose a fully convolutional neural network (CNN…
Clinical Center, NIH（美国国立卫生研究院临床中心）；National Center for Biotechnology Information, National Library of Medicine, NIH（美国国立卫生研究院国家医学图书馆国家生物技术信息中心）
Purpose: Identification of abdominal Lymph Nodes (LN) that are suspicious for metastasis in T2 Magnetic Resonance Imaging (MRI) scans is critical for staging of lymphoproliferative diseases. Prior work on LN detection has been limited to specific anatomical regions of the body (pelvis, rectum) in single MR slices. Therefore, the development of a universal approach to detect LN in full T2 MRI volumes is highly desirable. Methods: In this study, a Computer Aided Detection (CAD) pipeline to universally identify abdominal LN in volumetric T2 MRI using neural networks is proposed. First, we trained…
We propose a new generative adversarial architecture to mitigate imbalance data problem in medical image semantic segmentation where the majority of pixels belongs to a healthy region and few belong to lesion or non-health region. A model trained with imbalanced data tends to bias toward healthy data which is not desired in clinical applications and predicted outputs by these networks have high precision and low sensitivity. We propose a new conditional generative refinement network with three components: a generative, a discriminative, and a refinement network to mitigate unbalanced data prob…
Kharkevich Institute for Information Transmission Problems（哈尔凯维奇信息传输问题研究所）；Moscow Institute of Physics and Technology（莫斯科物理技术学院）；Moscow Gamma-Knife Center（莫斯科伽马刀中心）；Skolkovo Institute of Science and Technology（斯科尔科沃科学技术学院）；Burdenko Neurosurgery Institute（布尔坚科神经外科研究所）
Deep learning methods are actively used for brain lesion segmentation. One of the most popular models is DeepMedic, which was developed for segmentation of relatively large lesions like glioma and ischemic stroke. In our work, we consider segmentation of brain tumors appropriate to stereotactic radiosurgery which limits typical lesion sizes. These differences in target volumes lead to a large number of false negatives (especially for small lesions) as well as to an increased number of false positives for DeepMedic. We propose a new patch-sampling procedure to increase network performance for s…
All Sizes Matter: Improving Volumetric Brain Segmentation on Small Lesions
Technical University of Munich（慕尼黑工业大学）；Klinikum rechts der Isar（伊萨尔河右岸医院）；Deutsches Konsortium für Translationale Krebsforschung (DKTK)（德国转化癌症研究联盟（DKTK））；Helmholtz Center Munich（慕尼黑亥姆霍兹中心）
The brain tumor is the most aggressive kind of tumor and can cause low life expectancy if diagnosed at the later stages. Manual identification of brain tumors is tedious and prone to errors. Misdiagnosis can lead to false treatment and thus reduce the chances of survival for the patient. Medical resonance imaging (MRI) is the conventional method used to diagnose brain tumors and their types. This paper attempts to eliminate the manual process from the diagnosis process and use machine learning instead. We proposed the use of pretrained convolutional neural networks (CNN) for the diagnosis and …
Igor Sikorsky Kyiv Polytechnic Institute（伊戈尔·西科尔斯基基辅理工学院）
Images with abnormal brain anatomy produce problems for automatic segmentation techniques, and as a result poor ROI detection affects both quantitative measurements and visual assessment of perfusion data. This paper presents a new approach for fully automated and relatively accurate ROI detection from dynamic susceptibility contrast perfusion magnetic resonance and can therefore be applied excellently in the perfusion analysis. In the proposed approach the segmentation output is a binary mask of perfusion ROI that has zero values for air pixels, pixels that represent non-brain tissues, and ce…
Automated anatomy-based post-processing reduces false positives and improved interpretability of deep learning intracranial aneurysm detection
Brigham and Women’s Hospital（布列根和妇女医院）；Harvard Medical School（哈佛医学院）；Northeastern University（东北大学）；Dana-Farber Cancer Institute（丹娜-法伯癌症研究所）；Washing University in St. Louis（圣路易斯华盛顿大学）；First Affiliated Hospital of Guangzhou Medical University（广州医科大学附属第一医院）
Cornell University（康奈尔大学）；Weill Cornell Medical College（威尔康奈尔医学院）
Segmentation of anatomical regions of interest such as vessels or small lesions in medical images is still a difficult problem that is often tackled with manual input by an expert. One of the major challenges for this task is that the appearance of foreground (positive) regions can be similar to background (negative) regions. As a result, many automatic segmentation algorithms tend to exhibit asymmetric errors, typically producing more false positives than false negatives. In this paper, we aim to leverage this asymmetry and train a diverse ensemble of models with very high recall, while sacri…
Stanford University（斯坦福大学）；Oslo University Hospital（奥斯陆大学医院）
We introduce a novel ensembling method, Random Bundle (RB), that improves performance for brain metastases segmentation. We create our ensemble by training each network on our dataset with 50% of our annotated lesions censored out. We also apply a lopsided bootstrap loss to recover performance after inducing an in silico 50% false negative rate and make our networks more sensitive. We improve our network detection of lesions's mAP value by 39% and more than triple the sensitivity at 80% precision. We also show slight improvements in segmentation quality through DICE score. Further, RB ensembli…
Center for Biomedical Imaging, CIBM（生物医学成像中心（CIBM））
Brain aneurysm detection in Time-Of-Flight Magnetic Resonance Angiography (TOF-MRA) has undergone drastic improvements with the advent of Deep Learning (DL). However, performances of supervised DL models heavily rely on the quantity of labeled samples, which are extremely costly to obtain. Here, we present a DL model for aneurysm detection that overcomes the issue with ''weak'' labels: oversized annotations which are considerably faster to create. Our weak labels resulted to be four times faster to generate than their voxel-wise counterparts. In addition, our model leverages prior anatomical k…
Deep-Learning Atlas Registration for Melanoma Brain Metastases: Preserving Pathology While Enabling Cohort-Level Analyses
Department of Radiation Oncology, Medical Center – University of Freiburg, Faculty of Medicine, University of Freiburg（放射肿瘤科，弗赖堡大学医学中心-弗赖堡大学医学院）；Division of Medical Physics, Department of Radiation Oncology, Medical Center – University of Freiburg, Faculty of Medicine, University of Freiburg（医学物理系，放射肿瘤科，弗赖堡大学医学中心-弗赖堡大学医学院）；Department of Radiation Oncology, University Medical Center Schleswig-Holstein（放射肿瘤科，石勒苏益格-荷尔斯泰因大学医学中心）；Department of Radiation Oncology, TUM School of Medicine and Health, TUM University Hospital Rechts der Isar, Technical University of Munich（放射肿瘤科，慕尼黑技术大学医学院及健康科学系，慕尼黑技术大学医院雷chts der Isar）
BrainLesion Suite: A Flexible and User-Friendly Framework for Modular Brain Lesion Image Analysis
Department of Quantitative Biomedicine, University of Zurich, Zurich, Switzerland（苏黎世大学定量生物医学系）；Helmholtz AI, Helmholtz Zentrum München, Germany（海德堡人工智能，慕尼黑研究中心）；Department of Diagnostic and Interventional Neuroradiology, School of Medicine, Klinikum rechts der Isar, Technical University of Munich, Germany（医学诊断与介入神经放射学系，慕尼黑技术大学）；TranslaTUM - Central Institute for Translational Cancer Research, Technical University of Munich, Germany（TranslaTUM - 转化癌症研究中央研究所，慕尼黑技术大学）；Chair for AI in Medicine, Klinikum Rechts der Isar, Munich, Germany（医学人工智能系，慕尼黑Klinikum Rechts der Isar）；Department of Radiation Oncology, Klinikum rechts der Isar, Technical University of Munich, Germany（放射肿瘤学系，慕尼黑Klinikum Rechts der Isar）；Institute of Clinical Neuroimmunology, Faculty of Medicine, Ludwig Maximilian University, Munich, Germany（临床神经免疫学研究所，路德维希-马克西米利安大学）；School of Computation, Information and Technology, Technical University of Munich, Germany（计算、信息与技术学院，慕尼黑技术大学）；Division of Computational Pathology, Indiana School of Medicine, Indianapolis, IN, USA（计算病理学系，印第安纳医学院）；Research Center for Digital Technologies in Dentistry and CAD/CAM, Department of Dentistry, Faculty of Medicine and Dentistry, Danube Private University, Krems, Austria（牙科数字技术研究中心，医学与牙科学院，多瑙私人大学）；Medical Research Group, MLCommons, San Francisco, CA, USA（医学研究组，MLCommons，旧金山，美国）；Department of Neurology, Clinical Neuroscience Center and Brain Tumor Center, University and University Hospital Zurich, 8091 Zurich, Switzerland（神经学系，临床神经科学中心和脑肿瘤中心，苏黎世大学及苏黎世大学医院）；Department of Medical Radiation Physics, Stockholm University, Solna, Sweden（医学辐射物理学系，斯德哥尔摩大学）；Department of Oncology-Pathology, Karolinska Institutet, Solna, Sweden, Sweden（肿瘤病理学系，卡罗林斯卡研究所）；Division of Oncology and Children's Research Center, University Children's Hospital Zurich, Switzerland（肿瘤与儿童研究中心，苏黎世儿童医院）；Wallace H. Coulter Department of Biomedical Engineering at Georgia Tech and Emory University, Atlanta, USA（Wallace H. Coulter生物医学工程系，佐治亚理工学院和埃默里大学，美国）；Institute of Radiation Medicine (IRM), Helmholtz Zentrum München (HMGU), Germany（放射医学研究所（IRM），海德堡研究中心（HMGU））；German Consortium for Translational Cancer Research (DKTK), Partner Site Munich, Munich, Germany（德国转化癌症研究联盟（DKTK），慕尼黑合作伙伴站点）
A Total Variation Regularized Framework for Epilepsy-Related MRI Image Segmentation
Department of Computer Engineering, Modeling, Electronics and Systems (DIMES), University of Calabria, Rende 87036, Italy（计算机工程、建模、电子与系统系（DIMES），卡拉布里亚大学，Rende 87036，意大利）
University of Alberta（阿尔伯塔大学）；The Hospital for Sick Children（病童医院）；University of Toronto（多伦多大学）
White Matter Injury (WMI) is the most prevalent brain injury in the preterm neonate leading to developmental deficits. However, detecting WMI in Magnetic Resonance (MR) images of preterm neonate brains using traditional WM segmentation-based methods is difficult mainly due to lack of reliable preterm neonate brain atlases to guide segmentation. Hence, we propose a segmentation-free, fast, unsupervised, atlas-free WMI detection method. We detect the ventricles as blobs using a fast linear Maximally Stable Extremal Regions algorithm. A reference contour equidistant from the blobs and the brain-b…
Template matching by normalized cross correlation (NCC) is widely used for finding image correspondences. We improve the robustness of this algorithm by preprocessing images with "siamese" convolutional networks trained to maximize the contrast between NCC values of true and false matches. The improvement is quantified using patches of brain images from serial section electron microscopy. Relative to a parameter-tuned bandpass filter, siamese convolutional networks significantly reduce false matches. Furthermore, all false matches can be eliminated by removing a tiny fraction of all matches ba…
Indian Institute of Technology Madras（印度理工学院马德拉斯分校）；Sree Chitra Tirunal Institute for Medical Sciences and Technology（Sree Chitra Tirunal医学科学与技术研究所）
The work presented explores the use of denoising autoencoders (DAE) for brain lesion detection, segmentation and false positive reduction. Stacked denoising autoencoders (SDAE) were pre-trained using a large number of unlabeled patient volumes and fine tuned with patches drawn from a limited number of patients (n=20, 40, 65). The results show negligible loss in performance even when SDAE was fine tuned using 20 patients. Low grade glioma (LGG) segmentation was achieved using a transfer learning approach wherein a network pre-trained with High Grade Glioma (HGG) data was fine tuned using LGG im…
A cascade of fully convolutional neural networks is proposed to segment multi-modal Magnetic Resonance (MR) images with brain tumor into background and three hierarchical regions: whole tumor, tumor core and enhancing tumor core. The cascade is designed to decompose the multi-class segmentation problem into a sequence of three binary segmentation problems according to the subregion hierarchy. The whole tumor is segmented in the first step and the bounding box of the result is used for the tumor core segmentation in the second step. The enhancing tumor core is then segmented based on the boundi…
University of Engineering & Technology, Taxila（塔克西拉工程技术大学）
Detection of brain tumor using a segmentation based approach is critical in cases, where survival of a subject depends on an accurate and timely clinical diagnosis. Gliomas are the most commonly found tumors having irregular shape and ambiguous boundaries, making them one of the hardest tumors to detect. The automation of brain tumor segmentation remains a challenging problem mainly due to significant variations in its structure. An automated brain tumor segmentation algorithm using deep convolutional neural network (DCNN) is presented in this paper. A patch based approach along with an incept…
