The medical image fusion combines two or more modalities into a single view while medical image translation synthesizes new images and assists in data augmentation. Together, these methods help in faster diagnosis of high grade malignant gliomas. However, they might be untrustworthy due to which neurosurgeons demand a robust visualisation tool to verify the reliability of the fusion and translation results before they make pre-operative surgical decisions. In this paper, we propose a novel approach to compute a confidence heat map between the source-target image pair by estimating the information transfer from the source to the target image using the joint probability distribution of the two images. We evaluate several fusion and translation methods using our visualisation procedure and showcase its robustness in enabling neurosurgeons to make finer clinical decisions.
University of California San Francisco（加州大学旧金山分校）；University of Pennsylvania（宾夕法尼亚大学）
Here we present the University of California San Francisco Preoperative Diffuse Glioma MRI (UCSF-PDGM) dataset. The UCSF-PDGM dataset includes 500 subjects with histopathologically-proven diffuse gliomas who were imaged with a standardized 3 Tesla preoperative brain tumor MRI protocol featuring predominantly 3D imaging, as well as advanced diffusion and perfusion imaging techniques. The dataset also includes isocitrate dehydrogenase (IDH) mutation status for all cases and O6-methylguanine-DNA methyltransferase (MGMT) promotor methylation status for World Health Organization (WHO) grade III and…
A Comparison and Evaluation of Fine-tuned Convolutional Neural Networks to Large Language Models for Image Classification and Segmentation of Brain Tumors on MRI
Institute of Cancer Research（癌症研究所）；Universidad de Valladolid（巴利亚多利德大学）；AGH University of Science and Technology（AGH科技大学）；Cardiff University（卡迪夫大学）；Aberystwyth University（阿伯里斯特威斯大学）
Gliomas are brain tumours with a high mortality rate. There are various grades and sub-types of this tumour, and the treatment procedure varies accordingly. Clinicians and oncologists diagnose and categorise these tumours based on visual inspection of radiology and histology data. However, this process can be time-consuming and subjective. The computer-assisted methods can help clinicians to make better and faster decisions. In this paper, we propose a pipeline for automatic classification of gliomas into three sub-types: oligodendroglioma, astrocytoma, and glioblastoma, using both radiology a…
Washington University in St. Louis（圣路易斯华盛顿大学）；Mallinckrodt Institute of Radiology（马林克罗特放射研究所）；Washington University School of Medicine（华盛顿大学医学院）；Institute for Informatics（信息学研究所）
Isocitrate dehydrogenase (IDH) mutation and 1p/19q codeletion status are important prognostic markers for glioma. Currently, they are determined using invasive procedures. Our goal was to develop artificial intelligence-based methods to non-invasively determine these molecular alterations from MRI. For this purpose, pre-operative MRI scans of 2648 patients with gliomas (grade II-IV) were collected from Washington University School of Medicine (WUSM; n = 835) and publicly available datasets viz. Brain Tumor Segmentation (BraTS; n = 378), LGG 1p/19q (n = 159), Ivy Glioblastoma Atlas Project (Ivy…
Massachusetts General Hospital（麻省总医院）；Harvard University（哈佛大学）；General Electric Healthcare（通用电气医疗集团）；Indian Institute of Science（印度科学学院）
Gliomas are one of the most frequent brain tumors and are classified into high grade and low grade gliomas. The segmentation of various regions such as tumor core, enhancing tumor etc. plays an important role in determining severity and prognosis. Here, we have developed a multi-threshold model based on attention U-Net for identification of various regions of the tumor in magnetic resonance imaging (MRI). We propose a multi-path segmentation and built three separate models for the different regions of interest. The proposed model achieved mean Dice Coefficient of 0.59, 0.72, and 0.61 for enhan…
Towards Trustworthy AI for Glioma Diagnosis: A Task-Aware Evaluation of Uncertainty Quantification
Erasmus MC, University Medical Center Rotterdam（鹿特丹大学医学中心伊拉斯谟MC）；Amsterdam UMC, University of Amsterdam（阿姆斯特丹大学医学中心阿姆斯特丹UMC）；GE HealthCare（GE医疗）；Erasmus MC Cancer Institute（伊拉斯谟MC癌症研究所）；Medical Delta（医疗三角洲）
Applications of artificial intelligence in the analysis of histopathology images of gliomas: a review
Fraunhofer Institute for Digital Medicine MEVIS（弗劳恩霍夫数字医学研究所MEVIS）；Hannover Medical School（汉诺威医学院）；Medical Center - University of Freiburg（弗莱堡大学医学中心）；RWTH Aachen University Hospital（亚琛工业大学医院）；Klinikum Bremen-Mitte（不来梅中部医院）；Leibniz Institute for Prevention Research and Epidemiology - BIPS（莱布尼茨预防研究与流行病学研究所）；Bremen Cancer Registry（不来梅癌症登记处）；Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB（弗劳恩霍夫光电、系统技术及图像处理研究所IOSB）；German Heart Center Berlin（柏林德国心脏中心）
The most common primary brain tumors are gliomas, evolving from the cerebral supportive cells. For clinical follow-up, the evaluation of the preoperative tumor volume is essential. Volumetric assessment of tumor volume with manual segmentation of its outlines is a time-consuming process that can be overcome with the help of computerized segmentation methods. In this contribution, two methods for World Health Organization (WHO) grade IV glioma segmentation in the human brain are compared using magnetic resonance imaging (MRI) patient data from the clinical routine. One method uses balloon infla…
Targeted Neural Architectures in Multi-Objective Frameworks for Complete Glioma Characterization from Multimodal MRI
School of Computer Science and Engineering, Vellore Institute of Technology（韦洛尔理工学院计算机科学与工程学院）；College of Engineering, Guindy, Anna University（安娜大学金迪工程学院）；Institute of Neurosurgery, Madras Medical College（马德拉斯医学院神经外科研究所）
The Hospital for Sick Children（病童医院）；University of Toronto（多伦多大学）；Vector Institute（矢量研究所）
Brain tumor segmentation is a critical task for tumor volumetric analyses and AI algorithms. However, it is a time-consuming process and requires neuroradiology expertise. While there has been extensive research focused on optimizing brain tumor segmentation in the adult population, studies on AI guided pediatric tumor segmentation are scarce. Furthermore, MRI signal characteristics of pediatric and adult brain tumors differ, necessitating the development of segmentation algorithms specifically designed for pediatric brain tumors. We developed a segmentation model trained on magnetic resonance…
Multimodal Oncology Agent for IDH1 Mutation Prediction in Low-Grade Glioma
Department of Mathematics, Computer Science and Physics, University of Udine（乌迪大学数学、计算机科学与物理系）；Tissue Image Analytics Centre, Department of Computer Science, University of Warwick（沃里克大学计算机科学系组织图像分析中心）；Histofy Ltd（Histofy有限公司）
University of Minho（米尼奥大学）；University Hospital Inselspital（因塞尔施皮塔尔大学医院）；University of Bern（伯尔尼大学）
Glioblastoma Multiforme is a high grade, very aggressive, brain tumor, with patients having a poor prognosis. Lower grade gliomas are less aggressive, but they can evolve into higher grade tumors over time. Patient management and treatment can vary considerably with tumor grade, ranging from tumor resection followed by a combined radio- and chemotherapy to a "wait and see" approach. Hence, tumor grading is important for adequate treatment planning and monitoring. The gold standard for tumor grading relies on histopathological diagnosis of biopsy specimens. However, this procedure is invasive, …
Radiology Informatics Lab, Mayo Clinic（梅奥诊所放射信息实验室）；Department of Pathology, Mayo Clinic（梅奥诊所病理科）；Department of Neurologic Surgery, Mayo Clinic（梅奥诊所神经外科）
Objective: Several studies have associated codeletion of chromosome arms 1p/19q in low-grade gliomas (LGG) with positive response to treatment and longer progression free survival. Therefore, predicting 1p/19q status is crucial for effective treatment planning of LGG. In this study, we predict the 1p/19q status from MR images using convolutional neural networks (CNN), which could be a noninvasive alternative to surgical biopsy and histopathological analysis. Method: Our method consists of three main steps: image registration, tumor segmentation, and classification of 1p/19q status using CNN. W…
Resource-Efficient Glioma Segmentation on Sub-Saharan MRI
BAC School of Computing；Information Systems, Botswana Faculty of Medicine of Sfax, University of Sfax, Sfax, Tunisia Laboratory of Biophysics；Medical Technologies, Higher Institute of Medical Technologies of Tunis, University ElManar, Tunis, Tunisia London School of Hygiene；Tropical Medicine, London, UK Lawson Health Research Institute, London, Ontario, Canada Department of Radiology, Bayero University, Kano-Nigeria Department of Radiology, Aminu Kano Teaching Hospital, Kano-Nigeria Department of Anatomy, Federal University Lokoja college of health sciences, Kogi State, Nigeria Department of Computer Science；Information, College of Science, Majmaah University, AL-Majmaah, 11952, Saudi Arabia CES Laboratory, National School of Engineers of Sfax, Sfax, Tunisia Department of Computer Science；IT, Mulungushi University, Zambia Multimodal Imaging of Neurodegenerative Diseases (MiND) Lab, Department of Neurology；Neurosurgery, McGill University, Montreal, QC, Canada Department of Electrical；Computer Engineering, University of British Columbia, Vancouver, Canada Medical Artificial Intelligence Laboratory (MAI Lab), Lagos, Nigeria Montreal Neurological Institute, McGill University, Montréal, Canada Department of Medicine, University of Cape Town, South Africa Department of Clinical \& Radiation Oncology, University of Cape Town, South Africa
Recent analysis identified distinct genomic subtypes of lower-grade glioma tumors which are associated with shape features. In this study, we propose a fully automatic way to quantify tumor imaging characteristics using deep learning-based segmentation and test whether these characteristics are predictive of tumor genomic subtypes. We used preoperative imaging and genomic data of 110 patients from 5 institutions with lower-grade gliomas from The Cancer Genome Atlas. Based on automatic deep learning segmentations, we extracted three features which quantify two-dimensional and three-dimensional …
University Institute of Diagnostic and Interventional Neuroradiology（诊断与介入神经放射学大学研究所）；Inselspital, Bern University Hospital（伯尔尼大学医院 Inselspital）
The BraTS dataset contains a mixture of high-grade and low-grade gliomas, which have a rather different appearance: previous studies have shown that performance can be improved by separated training on low-grade gliomas (LGGs) and high-grade gliomas (HGGs), but in practice this information is not available at test time to decide which model to use. By contrast with HGGs, LGGs often present no sharp boundary between the tumor core and the surrounding edema, but rather a gradual reduction of tumor-cell density. Utilizing our 3D-to-2D fully convolutional architecture, DeepSCAN, which ranked highl…
Improving Deep Learning Models for Pediatric Low-Grade Glioma Tumors Molecular Subtype Identification Using 3D Probability Distributions of Tumor Location
The Hospital for Sick Children (SickKids)（SickKids 病童医院）；SickKids Research Institute（SickKids 研究所）；University of Toronto（多伦多大学）；Institute of Medical Science, University of Toronto（多伦多大学医学科学研究所）；Vector Institute（矢量研究院）；NVIDIA Deep Learning Institute（英伟达深度学习研究院）
School of Optoelectronic Science and Engineering, University of Electronic Science and Technology of China（电子科技大学光电科学与工程学院）；Institute of Medical Science, University of Toronto（多伦多大学医学科学研究所）；The Hospital for Sick Children (SickKids)（SickKids 儿童医院（病童医院））
Brain tumor is one of the leading causes of cancer-related death globally among children and adults. Precise classification of brain tumor grade (low-grade and high-grade glioma) at early stage plays a key role in successful prognosis and treatment planning. With recent advances in deep learning, Artificial Intelligence-enabled brain tumor grading systems can assist radiologists in the interpretation of medical images within seconds. The performance of deep learning techniques is, however, highly depended on the size of the annotated dataset. It is extremely challenging to label a large quanti…
Technical University of Munich（慕尼黑工业大学）；Sharif University of Technology（谢里夫理工大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
The need for a large amount of labeled data in the supervised setting has led recent studies to utilize self-supervised learning to pre-train deep neural networks using unlabeled data. Many self-supervised training strategies have been investigated especially for medical datasets to leverage the information available in the much fewer unlabeled data. One of the fundamental strategies in image-based self-supervision is context prediction. In this approach, a model is trained to reconstruct the contents of an arbitrary missing region of an image based on its surroundings. However, the existing m…
Carl Zeiss AG（卡尔蔡司股份公司）；University of Padua（帕多瓦大学）；University of Applied Sciences Western Switzerland (HES-SO Valais)（瑞士西部应用科学大学（瓦莱州高等专业学院））；Carl Zeiss Meditec AG（卡尔蔡司医疗科技股份公司）；IMEC（比利时微电子研究中心）；University Hospitals Leuven（鲁汶大学医院）；KU Leuven（鲁汶大学）
Surgery for gliomas (intrinsic brain tumors), especially when low-grade, is challenging due to the infiltrative nature of the lesion. Currently, no real-time, intra-operative, label-free and wide-field tool is available to assist and guide the surgeon to find the relevant demarcations for these tumors. While marker-based methods exist for the high-grade glioma case, there is no convenient solution available for the low-grade case; thus, marker-free optical techniques represent an attractive option. Although RGB imaging is a standard tool in surgical microscopes, it does not contain sufficient …
Deformable image registration is a key task in medical image analysis. The Brain Tumor Sequence Registration challenge (BraTS-Reg) aims at establishing correspondences between pre-operative and follow-up scans of the same patient diagnosed with an adult brain diffuse high-grade glioma and intends to address the challenging task of registering longitudinal data with major tissue appearance changes. In this work, we proposed a two-stage cascaded network based on the Inception and TransMorph models. The dataset for each patient was comprised of a native pre-contrast (T1), a contrast-enhanced T1-w…
Tumor Location-weighted MRI-Report Contrastive Learning: A Framework for Improving the Explainability of Pediatric Brain Tumor Diagnosis
The Hospital for Sick Children（病童医院）；University of Toronto（多伦多大学）；Vector Institute for Artificial Intelligence（矢量人工智能研究所）；University Hospital Augsburg（奥格斯堡大学医院）
Broad Institute of Harvard and MIT（哈佛大学与麻省理工学院博德研究所）；Dana-Farber Cancer Institute（丹娜法伯癌症研究院）；Department of Biomedical Informatics, Harvard Medical School（哈佛医学院生物医学信息学系）
Cancer diagnosis, prognosis, and therapeutic response predictions are based on morphological information from histology slides and molecular profiles from genomic data. However, most deep learning-based objective outcome prediction and grading paradigms are based on histology or genomics alone and do not make use of the complementary information in an intuitive manner. In this work, we propose Pathomic Fusion, an interpretable strategy for end-to-end multimodal fusion of histology image and genomic (mutations, CNV, RNA-Seq) features for survival outcome prediction. Our approach models pairwise…
MOAB: Multi-Modal Outer Arithmetic Block For Fusion Of Histopathological Images And Genetic Data For Brain Tumor Grading
School of Electronic Engineering and Computer Science, Queen Mary University of London（伦敦玛丽女王大学电子工程与计算机科学学院）；Queen Mary's Digital Environment Research Institute (DERI)（伦敦玛丽女王大学数字环境研究所（DERI））
Brain Tumor Classification on MRI in Light of Molecular Markers
Robotics Institute, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA Department of Electrical \& Computer Engineering, College of Engineering, Northeastern University, Boston, MA, USA University of Maryland, Baltimore County, MD, USA；Princeton University, Princeton NJ 08544, USA, , WWW home page: http://users/ iekeland/web/welcome.html Universit\' e de Paris-Sud, Laboratoire d'Analyse Num\' e rique, B\ a timent 425, F-91405 Orsay Cedex, France
Morphological attributes from histopathological images and molecular profiles from genomic data are important information to drive diagnosis, prognosis, and therapy of cancers. By integrating these heterogeneous but complementary data, many multi-modal methods are proposed to study the complex mechanisms of cancers, and most of them achieve comparable or better results from previous single-modal methods. However, these multi-modal methods are restricted to a single task (e.g., survival analysis or grade classification), and thus neglect the correlation between different tasks. In this study, w…
Yale University（耶鲁大学）；Medical University of Vienna（维也纳医科大学）；United Imaging Intelligence（联影智能）
Magnetic Resonance Spectroscopic Imaging (MRSI) is an essential tool for quantifying metabolites in the body, but the low spatial resolution limits its clinical applications. Deep learning-based super-resolution methods provided promising results for improving the spatial resolution of MRSI, but the super-resolved images are often blurry compared to the experimentally-acquired high-resolution images. Attempts have been made with the generative adversarial networks to improve the image visual quality. In this work, we consider another type of generative model, the flow-based model, of which the…
Towards Reliable Pediatric Brain Tumor Segmentation: Task-Specific nnU-Net Enhancements
Institute of Natural Sciences, School of Mathematical Sciences, MOE-LSC, Shanghai Jiao Tong University（自然科学院，数学科学学院，MOE-LSC，上海交通大学）；Department of Radiology, Huashan Hospital, Fudan University（放射科，华山医院，复旦大学）；Department of Neurosurgery, Huashan Hospital, Fudan University（神经外科，华山医院，复旦大学）
