Expectation-Maximization Regularized Deep Learning for Weakly Supervised Tumor Segmentation for Glioblastoma
University of Cambridge（剑桥大学）；Shanghai General Hospital（上海市第一人民医院）；Shanghai Jiao Tong University（上海交通大学）；Southern University of Science and Technology（南方科技大学）；University of Bath（巴斯大学）
Chao Li, Wenjian Huang, Xi Chen, Yiran Wei, Stephen J. Price, Carola-Bibiane Schönlieb
We present an Expectation-Maximization (EM) Regularized Deep Learning (EMReDL) model for weakly supervised tumor segmentation. The proposed framework is tailored to glioblastoma, a type of malignant tumor characterized by its diffuse infiltration into the surrounding brain tissue, which poses significant challenge to treatment target and tumor burden estimation using conventional structural MRI. Although physiological MRI provides more specific information regarding tumor infiltration, the relatively low resolution hinders a precise full annotation. This has motivated us to develop a weakly supervised deep learning solution that exploits the partial labelled tumor regions.
EMReDL contains two components: a physiological prior prediction model and EM-regularized segmentation model. The physiological prior prediction model exploits the physiological MRI by training a classifier to generate a physiological prior map. This map is passed to the segmentation model for regularization using the EM algorithm. We evaluated the model on a glioblastoma dataset with the pre-operative multiparametric and recurrence MRI available. EMReDL showed to effectively segment the infiltrated tumor from the partially labelled region of potential infiltration. The segmented core tumor and infiltrated tumor demonstrated high consistency with the tumor burden labelled by experts. The performance comparisons showed that EMReDL achieved higher accuracy than published state-of-the-art models. On MR spectroscopy, the segmented region displayed more aggressive features than other partial labelled region. The proposed model can be generalized to other segmentation tasks that rely on partial labels, with the CNN architecture flexible in the framework.
The Rio Hortega University Hospital Glioblastoma dataset: a comprehensive collection of preoperative, early postoperative and recurrence MRI scans (RHUH-GBM)
Glioblastoma, an aggressive brain cancer, is amongst the most lethal of all cancers. Expression of the O6-methylguanine-DNA-methyltransferase (MGMT) gene in glioblastoma tumor tissue is of clinical importance as it has a significant effect on the efficacy of Temozolomide, the primary chemotherapy treatment administered to glioblastoma patients. Currently, MGMT methylation is determined through an invasive brain biopsy and subsequent genetic analysis of the extracted tumor tissue. In this work, we present novel Bayesian classifiers that make probabilistic predictions of MGMT methylation status …
Case Western Reserve University（凯斯西储大学）；Cleveland Clinic（克利夫兰诊所）
A significant challenge in Glioblastoma (GBM) management is identifying pseudo-progression (PsP), a benign radiation-induced effect, from tumor recurrence, on routine imaging following conventional treatment. Previous studies have linked tumor lobar presence and laterality to GBM outcomes, suggesting that disease etiology and progression in GBM may be impacted by tumor location. Hence, in this feasibility study, we seek to investigate the following question: Can tumor location on treatment-naïve MRI provide early cues regarding likelihood of a patient developing pseudo-progression versus tumor…
Differentiating tumor progression (TP) from treatment-related necrosis (TN) is critical for clinical management decisions in glioblastoma (GBM). Dynamic FDG PET (dPET), an advance from traditional static FDG PET, may prove advantageous in clinical staging. dPET includes novel methods of a model-corrected blood input function that accounts for partial volume averaging to compute parametric maps that reveal kinetic information. In a preliminary study, a convolution neural network (CNN) was trained to predict classification accuracy between TP and TN for $35$ brain tumors from $26$ subjects in th…
Future Processing Healthcare（未来医疗处理公司）；Silesian University of Technology（西里西亚工业大学）；Medical University of Silesia in Katowice（卡托维兹西里西亚医科大学）；ZSM Chorzów（霍茹夫ZSM医院）；University Clinical Centre（大学临床中心）；Roche Pharmaceutical Research & Early Development（罗氏制药研究与早期开发部）
Tumor burden assessment by magnetic resonance imaging (MRI) is central to the evaluation of treatment response for glioblastoma. This assessment is complex to perform and associated with high variability due to the high heterogeneity and complexity of the disease. In this work, we tackle this issue and propose a deep learning pipeline for the fully automated end-to-end analysis of glioblastoma patients. Our approach simultaneously identifies tumor sub-regions, including the enhancing tumor, peritumoral edema and surgical cavity in the first step, and then calculates the volumetric and bidimens…
College of Electronic Science and Engineering, Jilin University（吉林大学电子科学与工程学院）；Wake Forest School of Medicine（维克森林医学院）
Pseudoprogression (PsP) occurs in 20-30% of patients with glioblastoma multiforme (GBM) after receiving the standard treatment. In the course of post-treatment magnetic resonance imaging (MRI), PsP exhibits similarities in shape and intensity to the true tumor progression (TTP) of GBM. So, these similarities pose challenges on the differentiation of these types of progression and hence the selection of the appropriate clinical treatment strategy. In this paper, we introduce DC-AL GAN, a novel feature learning method based on deep convolutional generative adversarial network (DCGAN) and AlexNet…
Glioblastomas are the most common malignant brain tumors in adults. Approximately 200000 people die each year from Glioblastoma in the world. Glioblastoma patients have a median survival of 12 months with optimal therapy and about 4 months without treatment. Glioblastomas appear as heterogeneous necrotic masses with irregular peripheral enhancement, surrounded by vasogenic edema. The current standard of care includes surgical resection, radiotherapy and chemotherapy, which require accurate segmentation of brain tumor subregions. For effective treatment planning, it is vital to identify the met…
The concept of tumor field effect implies that cancer is a systemic disease with its impact way beyond the visible tumor confines. For instance, in Glioblastoma (GBM), an aggressive brain tumor, the increase in intracranial pressure due to tumor burden often leads to brain herniation and poor outcomes. Our work is based on the rationale that highly aggressive tumors tend to grow uncontrollably, leading to pronounced biomechanical tissue deformations in the normal parenchyma, which when combined with local morphological differences in the tumor confines on MRI scans, will comprehensively captur…
Department of Health Technology, Technical University of Denmark（丹麦技术大学健康技术系）；Department of Oncology, The Finsen Center, Rigshospitalet（哥本哈根大学医院芬森中心肿瘤学系）；Department of Clinical Physiology, Nuclear Medicine and PET, Center of Diagnostic Investigation, Rigshospitalet（哥本哈根大学医院诊断研究中心临床生理学、核医学与正电子发射断层扫描科）；Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School（哈佛医学院麻省总医院Athinoula A. Martinos生物医学成像中心）
Survival prediction models can potentially be used to guide treatment of glioblastoma patients. However, currently available MR imaging biomarkers holding prognostic information are often challenging to interpret, have difficulties generalizing across data acquisitions, or are only applicable to pre-operative MR data. In this paper we aim to address these issues by introducing novel imaging features that can be automatically computed from MR images and fed into machine learning models to predict patient survival. The features we propose have a direct biological interpretation: They measure the…
Decoding MGMT Methylation: A Step Towards Precision Medicine in Glioblastoma
National University of Computer and Emerging Sciences（国立计算机与新兴科学大学）；Oryx Universal College with Liverpool John Moores University（羚羊环球学院与利物浦约翰摩尔斯大学）；Al-Balqa Applied University（巴尔卡应用大学）
Glioblastoma is profoundly heterogeneous in microstructure and vasculature, which may lead to tumor regional diversity and distinct treatment response. Although successful in tumor sub-region segmentation and survival prediction, radiomics based on machine learning algorithms, is challenged by its robustness, due to the vague intermediate process and track changes. Also, the weak interpretability of the model poses challenges to clinical application. Here we proposed a machine learning framework to semi-automatically fine-tune the clustering algorithms and quantitatively identify stable sub-re…
Glioblastoma Multiforme is a very aggressive type of brain tumor. Due to spatial and temporal intra-tissue inhomogeneity, location and the extent of the cancer tissue, it is difficult to detect and dissect the tumor regions. In this paper, we propose survival prognosis models using four regressors operating on handcrafted image-based and radiomics features. We hypothesize that the radiomics shape features have the highest correlation with survival prediction. The proposed approaches were assessed on the Brain Tumor Segmentation (BraTS-2020) challenge dataset. The highest accuracy of image feat…
Reutlingen University（罗伊特林根大学）；Menoufia University（梅努菲亚大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
Glioblastomas are the most aggressive fast-growing primary brain cancer which originate in the glial cells of the brain. Accurate identification of the malignant brain tumor and its sub-regions is still one of the most challenging problems in medical image segmentation. The Brain Tumor Segmentation Challenge (BraTS) has been a popular benchmark for automatic brain glioblastomas segmentation algorithms since its initiation. In this year, BraTS 2021 challenge provides the largest multi-parametric (mpMRI) dataset of 2,000 pre-operative patients. In this paper, we propose a new aggregation of two …
Towards a deep learning approach for classifying treatment response in glioblastomas
Institute for Systems and Robotics - Lisbon and Department of Bioengineering of Instituto Superior Técnico（系统与机器人研究所 - 里斯本及 Instituto Superior Técnico 生物工程系）；Hospital da Luz Learning Health（Luz医院学习健康中心）；Imaging Department, Hospital da Luz（Luz医院影像部门）
Postoperative glioblastoma segmentation: Development of a fully automated pipeline using deep convolutional neural networks and comparison with currently available models
Río Hortega University Hospital（里奥·奥尔特加大学医院）；Universidad de Valladolid（巴利亚多利德大学）；Center for Biomedical Research in Network of Bioengineering, Biomaterials and Nanomedicine (CIBER-BBN)（生物工程、生物材料和纳米医学网络生物医学研究中心）；Albacete University Hospital（阿尔瓦塞特大学医院）；La Princesa University Hospital（拉普林塞萨大学医院）；UiT The Arctic University of Norway（挪威北极大学）；University Hospital of North Norway（北挪威大学医院）；St. Olavs University Hospital（圣奥拉夫大学医院）；Norwegian University of Science and Technology（挪威科技大学）；St. Olavs Hospital, Trondheim University Hospital（圣奥拉夫医院，特隆赫姆大学医院）；de Octubre University Hospital (i + 12)（10月12日大学医院）；Complutense University（马德里康普顿斯大学）；Instituto de Investigación Sanitaria, 12 de Octubre University Hospital (i + 12)（10月12日大学医院卫生研究所）；Institute for Research in Mathematics (IMUVA), University of Valladolid（巴利亚多利德大学数学研究所）
University of California at Riverside（加州大学河滨分校）；Emory University（埃默里大学）；Riverside Community Hospital（河滨社区医院）
While deep learning approaches have shown remarkable performance in many imaging tasks, most of these methods rely on availability of large quantities of data. Medical image data, however, is scarce and fragmented. Generative Adversarial Networks (GANs) have recently been very effective in handling such datasets by generating more data. If the datasets are very small, however, GANs cannot learn the data distribution properly, resulting in less diverse or low-quality results. One such limited dataset is that for the concurrent gain of 19 and 20 chromosomes (19/20 co-gain), a mutation with posit…
Patient-specific prediction of glioblastoma growth via reduced order modeling and neural networks
Mathematics Area, mathLab, SISSA（SISSA数学区mathLab）；Dipartimento di Matematica e Fisica “N. Tartaglia”, Università Cattolica del Sacro Cuore（圣心大学N. Tartaglia数学与物理系）；Department of Neurosurgery – Fondazione I.R.C.C.S. Istituto Neurologico Carlo Besta（卡尔·贝斯塔神经科学研究所神经外科部）
We proposed a fully automatic workflow for glioblastoma (GBM) survival prediction using deep learning (DL) methods. 285 glioma (210 GBM, 75 low-grade glioma) patients were included. 163 of the GBM patients had overall survival (OS) data. Every patient had four pre-operative MR scans and manually drawn tumor contours. For automatic tumor segmentation, a 3D convolutional neural network (CNN) was trained and validated using 122 glioma patients. The trained model was applied to the remaining 163 GBM patients to generate tumor contours. The handcrafted and DL-based radiomic features were extracted …
Segmentation of glioblastomas in early post-operative multi-modal MRI with deep neural networks
SINTEF Digital（SINTEF数字研究所）；Norwegian University of Science and Technology（挪威科技大学）；Amsterdam University Medical Centers（阿姆斯特丹大学医学中心）；Twee Steden Hospital（双城医院）；University College London（伦敦大学学院）
Convolutional neural networks have achieved excellent results in automatic medical image segmentation. In this study, we proposed a novel 3D multi-path DenseNet for generating the accurate glioblastoma (GBM) tumor contour from four multi-modal pre-operative MR images. We hypothesized that the multi-path architecture could achieve more accurate segmentation than a single-path architecture. 258 GBM patients were included in this study. Each patient had four MR images (T1-weighted, contrast-enhanced T1-weighted, T2-weighted, and FLAIR) and the manually segmented tumor contour. We built a 3D multi…
Raidionics: an open software for pre- and postoperative central nervous system tumor segmentation and standardized reporting
SINTEF Digital（挪威科技工业研究院数字分院）；Norwegian University of Science and Technology (NTNU)（挪威科技大学（NTNU））；St. Olavs hospital, Trondheim University Hospital（特隆赫姆大学医院圣奥拉夫斯医院）
National University of Singapore（新加坡国立大学）；University of Moratuwa（莫拉图瓦大学）
The accurate prognosis of Glioblastoma Multiforme (GBM) plays an essential role in planning correlated surgeries and treatments. The conventional models of survival prediction rely on radiomic features using magnetic resonance imaging (MRI). In this paper, we propose a radiogenomic overall survival (OS) prediction approach by incorporating gene expression data with radiomic features such as shape, geometry, and clinical information. We exploit TCGA (The Cancer Genomic Atlas) dataset and synthesize the missing MRI modalities using a fully convolutional network (FCN) in a conditional Generative …
Mohamed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Glioblastoma is a common brain malignancy that tends to occur in older adults and is almost always lethal. The effectiveness of chemotherapy, being the standard treatment for most cancer types, can be improved if a particular genetic sequence in the tumor known as MGMT promoter is methylated. However, to identify the state of the MGMT promoter, the conventional approach is to perform a biopsy for genetic analysis, which is time and effort consuming. A couple of recent publications proposed a connection between the MGMT promoter state and the MRI scans of the tumor and hence suggested the use o…
Automated Ensemble-Based Segmentation of Adult Brain Tumors: A Novel Approach Using the BraTS AFRICA Challenge Data
Manning College of Information and Computer Sciences, University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校曼宁信息与计算机科学学院）；Baystate Medical Center（贝斯州医疗中心）；UMass Chan Medical School - Baystate Campus（马萨诸塞大学陈医学院贝斯州校区）
本文面向撒哈拉以南非洲成人患者，提出融合 UNet3D、ONet3D、SphereNet3D 及改进损失函数的十一模型集成脑肿瘤分割方法，在 BraTS AFRICA 数据上取得优于单模型的分割性能。
Our objective is the calibration of mathematical tumor growth models from a single multiparametric scan. The target problem is the analysis of preoperative Glioblastoma (GBM) scans. To this end, we present a fully automatic tumor-growth calibration methodology that integrates a single-species reaction-diffusion partial differential equation (PDE) model for tumor progression with multiparametric Magnetic Resonance Imaging (mpMRI) scans to robustly extract patient specific biomarkers i.e., estimates for (i) the tumor cell proliferation rate, (ii) the tumor cell migration rate, and (iii) the orig…
Re-DiffiNet: Modeling discrepancies in tumor segmentation using diffusion models
University of Washington（华盛顿大学）；Paul G. Allen School of Computer Science, University of Washington（华盛顿大学保罗·G·艾伦计算机科学学院）；Department of Neurological Surgery, University of Washington（华盛顿大学神经外科系）
Glioma is a common type of brain tumor, and accurate detection of it plays a vital role in the diagnosis and treatment process. Despite advances in medical image analyzing, accurate tumor segmentation in brain magnetic resonance (MR) images remains a challenge due to variations in tumor texture, position, and shape. In this paper, we propose a fast, automated method, with light computational complexity, to find the smallest bounding box around the tumor region. This region-of-interest can be used as a preprocessing step in training networks for subregion tumor segmentation. By adopting the out…
We present the SIBIA (Scalable Integrated Biophysics-based Image Analysis) framework for joint image registration and biophysical inversion and we apply it to analyse MR images of glioblastomas (primary brain tumors). In particular, we consider the following problem. Given the segmentation of a normal brain MRI and the segmentation of a cancer patient MRI, we wish to determine tumor growth parameters and a registration map so that if we "grow a tumor" (using our tumor model) in the normal segmented image and then register it to the patient segmented image, then the registration mismatch is as …
Case Western Reserve University（凯斯西储大学）；Cleveland Clinic（克利夫兰诊所）
With growing emphasis on personalized cancer-therapies,radiogenomics has shown promise in identifying target tumor mutational status on routine imaging (i.e. MRI) scans. These approaches fall into 2 categories: (1) deep-learning/radiomics (context-based), using image features from the entire tumor to identify the gene mutation status, or (2) atlas (spatial)-based to obtain likelihood of gene mutation status based on population statistics. While many genes (i.e. EGFR, MGMT) are spatially variant, a significant challenge in reliable assessment of gene mutation status on imaging has been the lack…
Preoperative brain tumor imaging: models and software for segmentation and standardized reporting
SINTEF Digital（挪威科技工业研究院数字研究所）；Norwegian University of Science and Technology（挪威科技大学）；St. Olavs hospital, Trondheim University Hospital（特隆赫姆大学医院圣奥拉夫医院）；Sahlgrenska University Hospital（萨尔格伦斯卡大学医院）；Sahlgrenska Academy, University of Gothenburg（哥德堡大学萨尔格伦斯卡学院）；Oslo University Hospital（奥斯陆大学医院）；Amsterdam University Medical Centers, Vrije Universiteit（阿姆斯特丹自由大学阿姆斯特丹大学医学中心）；Cancer Center Amsterdam, Brain Tumor Center, Amsterdam University Medical Centers（阿姆斯特丹大学医学中心阿姆斯特丹癌症中心脑肿瘤中心）
Hôpital Erasme（埃拉斯姆医院）；Université libre de Bruxelles（布鲁塞尔自由大学）；Center for Microscopy and Molecular Imaging（显微与分子成像中心）
Diffuse gliomas are highly infiltrative tumors whose early diagnosis and follow-up usually rely on magnetic resonance imaging (MRI). However, the limited sensitivity of this technique makes it impossible to directly assess the extent of the glioma cell invasion, leading to sub-optimal treatment planing. Reaction-diffusion growth models have been proposed for decades to extrapolate glioma cell infiltration beyond margins visible on MRI and predict its spatial-temporal evolution. These models nevertheless require an initial condition, that is the tumor cell density values at every location of th…
Washington University in St. Louis（圣路易斯华盛顿大学）；Mallinckrodt Institute of Radiology（马林克罗特放射研究所）；Washington University School of Medicine（华盛顿大学医学院）；Institute for Informatics（信息学研究所）
Isocitrate dehydrogenase (IDH) mutation and 1p/19q codeletion status are important prognostic markers for glioma. Currently, they are determined using invasive procedures. Our goal was to develop artificial intelligence-based methods to non-invasively determine these molecular alterations from MRI. For this purpose, pre-operative MRI scans of 2648 patients with gliomas (grade II-IV) were collected from Washington University School of Medicine (WUSM; n = 835) and publicly available datasets viz. Brain Tumor Segmentation (BraTS; n = 378), LGG 1p/19q (n = 159), Ivy Glioblastoma Atlas Project (Ivy…
