Cross-Modality Deep Feature Learning for Brain Tumor Segmentation
School of Mechano-Electronic Engineering, Xidian University（西安电子科技大学机电工程学院）；Computer Science Department, Aberystwyth University（阿伯里斯特威斯大学计算机科学系）；School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；Deepwise AI Lab（深睿人工智能实验室）
Recent advances in machine learning and prevalence of digital medical images have opened up an opportunity to address the challenging brain tumor segmentation (BTS) task by using deep convolutional neural networks. However, different from the RGB image data that are very widespread, the medical image data used in brain tumor segmentation are relatively scarce in terms of the data scale but contain the richer information in terms of the modality property. To this end, this paper proposes a novel cross-modality deep feature learning framework to segment brain tumors from the multi-modality MRI data. The core idea is to mine rich patterns across the multi-modality data to make up for the insufficient data scale. The proposed cross-modality deep feature learning framework consists of two learning processes: the cross-modality feature transition (CMFT) process and the cross-modality feature fusion (CMFF) process, which aims at learning rich feature representations by transiting knowledge across different modality data and fusing knowledge from different modality data, respectively. Comprehensive experiments are conducted on the BraTS benchmarks, which show that the proposed cross-modality deep feature learning framework can effectively improve the brain tumor segmentation performance when compared with the baseline methods and state-of-the-art methods.
Isfahan University of Technology（伊斯法罕理工大学）；Seattle University（西雅图大学）；McMaster University（麦克马斯特大学）
According to the World Health Organization (WHO), cancer is the second leading cause of death worldwide, responsible for over 9.5 million deaths in 2018 alone. Brain tumors count for one out of every four cancer deaths. Therefore, accurate and timely diagnosis of brain tumors will lead to more effective treatments. Physicians classify brain tumors only with biopsy operation by brain surgery, and after diagnosing the type of tumor, a treatment plan is considered for the patient. Automatic systems based on machine learning algorithms can allow physicians to diagnose brain tumors with noninvasive…
Vingroup Big Data Institute (VinBigdata)（越南 VinBigdata 大数据研究院）；Hanoi University of Science and Technology (HUST)（河内科技大学）；University of Engineering and Technology (UET), VNU（越南国立大学工程技术大学）
Brain tumor segmentation plays an essential role in medical image analysis. In recent studies, deep convolution neural networks (DCNNs) are extremely powerful to tackle tumor segmentation tasks. We propose in this paper a novel training method that enhances the segmentation results by adding an additional classification branch to the network. The whole network was trained end-to-end on the Multimodal Brain Tumor Segmentation Challenge (BraTS) 2020 training dataset. On the BraTS's validation set, it achieved an average Dice score of 78.43%, 89.99%, and 84.22% respectively for the enhancing tumo…
University of Electronic Science and Technology of China（电子科技大学）；New Jersey Institute of Technology（新泽西理工学院）
Past few years have witnessed the prevalence of deep learning in many application scenarios, among which is medical image processing. Diagnosis and treatment of brain tumors requires an accurate and reliable segmentation of brain tumors as a prerequisite. However, such work conventionally requires brain surgeons significant amount of time. Computer vision techniques could provide surgeons a relief from the tedious marking procedure. In this paper, a 3D U-net based deep learning model has been trained with the help of brain-wise normalization and patching strategies for the brain tumor segmenta…
Huazhong University of Science and Technology（华中科技大学）；Shenzhen University（深圳大学）
Deep convolutional neural network (CNN) achieves remarkable performance for medical image analysis. UNet is the primary source in the performance of 3D CNN architectures for medical imaging tasks, including brain tumor segmentation. The skip connection in the UNet architecture concatenates features from both encoder and decoder paths to extract multi-contextual information from image data. The multi-scaled features play an essential role in brain tumor segmentation. However, the limited use of features can degrade the performance of the UNet approach for segmentation. In this paper, we propose…
Glioma is a common type of brain tumor, and accurate detection of it plays a vital role in the diagnosis and treatment process. Despite advances in medical image analyzing, accurate tumor segmentation in brain magnetic resonance (MR) images remains a challenge due to variations in tumor texture, position, and shape. In this paper, we propose a fast, automated method, with light computational complexity, to find the smallest bounding box around the tumor region. This region-of-interest can be used as a preprocessing step in training networks for subregion tumor segmentation. By adopting the out…
BrainNormalizer: Anatomy-Informed Pseudo-Healthy Brain Reconstruction from Tumor MRI via Edge-Guided ControlNet
University of Pittsburgh（匹兹堡大学）；Georgia Institute of Technology（佐治亚理工学院）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；Cedars-Sinai Medical Center（西德萨斯医疗中心）；Mayo Clinic Arizona（梅奥诊所亚利桑那分部）
Korea Advanced Institute of Science and Technology (KAIST)（韩国高等科学技术研究院（KAIST））
Medical image analysis using deep neural networks has been actively studied. Deep neural networks are trained by learning data. For accurate training of deep neural networks, the learning data should be sufficient, of good quality, and should have a generalized property. However, in medical images, it is difficult to acquire sufficient patient data because of the difficulty of patient recruitment, the burden of annotation of lesions by experts, and the invasion of patients' privacy. In comparison, the medical images of healthy volunteers can be easily acquired. Using healthy brain images, the …
Brain Tumor Classification using Vision Transformer with Selective Cross-Attention Mechanism and Feature Calibration
K.N. Toosi University of Technology（霍梅尼国际大学（K.N. Toosi理工大学））；Indiana University-Purdue University（印第安纳大学-普渡大学）；Shahed university（沙希德大学）；University of Cincinnati（辛辛那提大学）
Shandong Normal University（山东师范大学）；Western University（韦仕敦大学）
Simultaneous segmentation and detection of liver tumors (hemangioma and hepatocellular carcinoma (HCC)) by using multi-modality non-contrast magnetic resonance imaging (NCMRI) are crucial for the clinical diagnosis. However, it is still a challenging task due to: (1) the HCC information on NCMRI is invisible or insufficient makes extraction of liver tumors feature difficult; (2) diverse imaging characteristics in multi-modality NCMRI causes feature fusion and selection difficult; (3) no specific information between hemangioma and HCC on NCMRI cause liver tumors detection difficult. In this stu…
We explore encoding brain symmetry into a neural network for a brain tumor segmentation task. A healthy human brain is symmetric at a high level of abstraction, and the high-level asymmetric parts are more likely to be tumor regions. Paying more attention to asymmetries has the potential to boost the performance in brain tumor segmentation. We propose a method to encode brain symmetry into existing neural networks and apply the method to a state-of-the-art neural network for medical imaging segmentation. We evaluate our symmetry-encoded network on the dataset from a brain tumor segmentation ch…
University of Calcutta（加尔各答大学）；West Bengal University of Technology（西孟加拉技术大学）
Tumor segmentation from magnetic resonance imaging (MRI) data is an important but time consuming manual task performed by medical experts. Automating this process is a challenging task because of the high diversity in the appearance of tumor tissues among different patients and in many cases similarity with the normal tissues. MRI is an advanced medical imaging technique providing rich information about the human soft-tissue anatomy. There are different brain tumor detection and segmentation methods to detect and segment a brain tumor from MRI images. These detection and segmentation approache…
National University of Computer and Emerging Sciences（国立计算机与新兴科学大学）；MD ANDERSON Cancer Center, The University of Texas（德克萨斯大学MD安德森癌症中心）；Hamad Bin Khalifa University（哈马德·本·哈利法大学）
Generative models have been very successful over the years and have received significant attention for synthetic data generation. As deep learning models are getting more and more complex, they require large amounts of data to perform accurately. In medical image analysis, such generative models play a crucial role as the available data is limited due to challenges related to data privacy, lack of data diversity, or uneven data distributions. In this paper, we present a method to generate brain tumor MRI images using generative adversarial networks. We have utilized StyleGAN2 with ADA methodol…
Brain tumor segmentation is a challenging problem in medical image analysis. The endpoint is to generate the salient masks that accurately identify brain tumor regions in an fMRI screening. In this paper, we propose a novel attention gate (AG model) for brain tumor segmentation that utilizes both the edge detecting unit and the attention gated network to highlight and segment the salient regions from fMRI images. This feature enables us to eliminate the necessity of having to explicitly point towards the damaged area(external tissue localization) and classify(classification) as per classical c…
Hybrid Multihead Attentive Unet-3D for Brain Tumor Segmentation
University of Engineering and Technology, Lahore（拉合尔工程技术大学）；School of Interdisciplinary Engineering and Sciences, National University of Sciences and Technology（国家科技大学跨学科工程与科学学院）
University College Dublin（都柏林大学学院）；Université Paris-Est Créteil Val de Marne - Université Paris 12（巴黎东-马恩拉瓦莱大学（巴黎第十二大学））；ADAPT SFI Research Centre（ADAPT SFI研究中心）
Magnetic Resonance Imaging (MRI) is the most commonly used non-intrusive technique for medical image acquisition. Brain tumor segmentation is the process of algorithmically identifying tumors in brain MRI scans. While many approaches have been proposed in the literature for brain tumor segmentation, this paper proposes a lightweight implementation of U-Net. Apart from providing real-time segmentation of MRI scans, the proposed architecture does not need large amount of data to train the proposed lightweight U-Net. Moreover, no additional data augmentation step is required. The lightweight U-Ne…
University of Hyogo（兵库大学）；Vanderbilt University（范德堡大学）；BRAC University（BRAC大学）
Diagnosing Brain Tumor with the aid of Magnetic Resonance Imaging (MRI) has gained enormous prominence over the years, primarily in the field of medical science. Detection and/or partitioning of brain tumors solely with the aid of MR imaging is achieved at the cost of immense time and effort and demands a lot of expertise from engaged personnel. This substantiates the necessity of fabricating an autonomous model brain tumor diagnosis. Our work involves implementing a deep convolutional neural network (DCNN) for diagnosing brain tumors from MR images. The dataset used in this paper consists of …
Brain tumor segmentation from magnetic resonance imaging (MRI) plays an important role in diagnostic radiology. To overcome the practical issues in manual approaches, there is a huge demand for building automatic tumor segmentation algorithms. This work introduces an efficient brain tumor summation model by exploiting the advancement in MRI and graph neural networks (GNNs). The model represents the volumetric MRI as a region adjacency graph (RAG) and learns to identify the type of tumors through a graph attention network (GAT) -- a variant of GNNs. The ablation analysis conducted on two benchm…
Pakistan Institute of Engineering and Applied Sciences（巴基斯坦工程与应用科学学院）
Brain tumors analysis is important in timely diagnosis and effective treatment to cure patients. Tumor analysis is challenging because of tumor morphology like size, location, texture, and heteromorphic appearance in the medical images. In this regard, a novel two-phase deep learning-based framework is proposed to detect and categorize brain tumors in magnetic resonance images (MRIs). In the first phase, a novel deep boosted features and ensemble classifiers (DBF-EC) scheme is proposed to detect tumor MRI images from healthy individuals effectively. The deep boosted feature space is achieved t…
University of California, Los Angeles（加州大学洛杉矶分校）；National University of Singapore（新加坡国立大学）
Brain tumor is a common and fatal form of cancer which affects both adults and children. The classification of brain tumors into different types is hence a crucial task, as it greatly influences the treatment that physicians will prescribe. In light of this, medical imaging techniques, especially those applying deep convolutional networks followed by a classification layer, have been developed to make possible computer-aided classification of brain tumor types. In this paper, we present a novel approach of directly learning deep embeddings for brain tumor types, which can be used for downstrea…
Automatic MRI brain tumor segmentation is of vital importance for the disease diagnosis, monitoring, and treatment planning. In this paper, we propose a two-stage encoder-decoder based model for brain tumor subregional segmentation. Variational autoencoder regularization is utilized in both stages to prevent the overfitting issue. The second-stage network adopts attention gates and is trained additionally using an expanded dataset formed by the first-stage outputs. On the BraTS 2020 validation dataset, the proposed method achieves the mean Dice score of 0.9041, 0.8350, and 0.7958, and Hausdorf…
Pakistan Institute of Engineering and Applied Sciences (PIEAS)（巴基斯坦工程与应用科学学院）；Ibadat International University（伊巴达特国际大学）；University of Engineering and Applied Science (UEAS)（工程与应用科学大学）
Brain tumor classification is crucial for clinical analysis and an effective treatment plan to cure patients. Deep learning models help radiologists to accurately and efficiently analyze tumors without manual intervention. However, brain tumor analysis is challenging because of its complex structure, texture, size, location, and appearance. Therefore, a novel deep residual and regional-based Res-BRNet Convolutional Neural Network (CNN) is developed for effective brain tumor (Magnetic Resonance Imaging) MRI classification. The developed Res-BRNet employed Regional and boundary-based operations …
Tumor Location-weighted MRI-Report Contrastive Learning: A Framework for Improving the Explainability of Pediatric Brain Tumor Diagnosis
The Hospital for Sick Children（病童医院）；University of Toronto（多伦多大学）；Vector Institute for Artificial Intelligence（矢量人工智能研究所）；University Hospital Augsburg（奥格斯堡大学医院）
Concordia Institute for Information Systems Engineering, Concordia University（康考迪亚大学康考迪亚信息系统工程研究所）；University of Toronto（多伦多大学）
According to official statistics, cancer is considered as the second leading cause of human fatalities. Among different types of cancer, brain tumor is seen as one of the deadliest forms due to its aggressive nature, heterogeneous characteristics, and low relative survival rate. Determining the type of brain tumor has significant impact on the treatment choice and patient's survival. Human-centered diagnosis is typically error-prone and unreliable resulting in a recent surge of interest to automatize this process using convolutional neural networks (CNNs). CNNs, however, fail to fully utilize …
School of Computer Science, University of Lincoln（林肯大学计算机科学学院）；National Heart & Lung Institute, Imperial College London（帝国理工学院国家心肺研究所）
In this paper, we propose a novel learning based method for automated segmentation of brain tumor in multimodal MRI images, which incorporates two sets of machine -learned and hand crafted features. Fully convolutional networks (FCN) forms the machine learned features and texton based features are considered as hand-crafted features. Random forest (RF) is used to classify the MRI image voxels into normal brain tissues and different parts of tumors, i.e. edema, necrosis and enhancing tumor. The method was evaluated on BRATS 2017 challenge dataset. The results show that the proposed method provi…
University of Arkansas（阿肯色大学）；Concordia University（康考迪亚大学）；Carnegie Mellon University（卡内基梅隆大学）
In recent years, deep neural networks have achieved state-of-the-art performance in a variety of recognition and segmentation tasks in medical imaging including brain tumor segmentation. We investigate that segmenting a brain tumor is facing to the imbalanced data problem where the number of pixels belonging to the background class (non tumor pixel) is much larger than the number of pixels belonging to the foreground class (tumor pixel). To address this problem, we propose a multi-task network which is formed as a cascaded structure. Our model consists of two targets, i.e., (i) effectively dif…
One of the most important tasks in medical image processing is the brain's whole tumor segmentation. It assists in quicker clinical assessment and early detection of brain tumors, which is crucial for lifesaving treatment procedures of patients. Because, brain tumors often can be malignant or benign, if they are detected at an early stage. A brain tumor is a collection or a mass of abnormal cells in the brain. The human skull encloses the brain very rigidly and any growth inside this restricted place can cause severe health issues. The detection of brain tumors requires careful and intricate a…
Massachusetts General Hospital（麻省总医院）；Harvard University（哈佛大学）；General Electric Healthcare（通用电气医疗集团）；Indian Institute of Science（印度科学学院）
Gliomas are one of the most frequent brain tumors and are classified into high grade and low grade gliomas. The segmentation of various regions such as tumor core, enhancing tumor etc. plays an important role in determining severity and prognosis. Here, we have developed a multi-threshold model based on attention U-Net for identification of various regions of the tumor in magnetic resonance imaging (MRI). We propose a multi-path segmentation and built three separate models for the different regions of interest. The proposed model achieved mean Dice Coefficient of 0.59, 0.72, and 0.61 for enhan…
Beijing-Dublin International College（北京-都柏林国际学院）；University College Dublin（都柏林大学）
This technical report presents a comparative analysis of existing deep learning (DL) based approaches for brain tumor segmentation with missing MRI modalities. Approaches evaluated include the Adversarial Co-training Network (ACN) and a combination of mmGAN and DeepMedic. A more stable and easy-to-use version of mmGAN is also open-sourced at a GitHub repository. Using the BraTS2018 dataset, this work demonstrates that the state-of-the-art ACN performs better especially when T1c is missing. While a simple combination of mmGAN and DeepMedic also shows strong potentials when only one MRI modality…
Concordia Institute for Information Systems Engineering, Concordia University（康考迪亚大学康考迪亚信息系统工程研究所）；University of Toronto（多伦多大学）
Brain tumor is considered as one of the deadliest and most common form of cancer both in children and in adults. Consequently, determining the correct type of brain tumor in early stages is of significant importance to devise a precise treatment plan and predict patient's response to the adopted treatment. In this regard, there has been a recent surge of interest in designing Convolutional Neural Networks (CNNs) for the problem of brain tumor type classification. However, CNNs typically require large amount of training data and can not properly handle input transformations. Capsule networks (r…
South China University of Technology（华南理工大学）；Rochester Institute of Technology（罗切斯特理工学院）
Due to the difficulties of obtaining multimodal paired images in clinical practice, recent studies propose to train brain tumor segmentation models with unpaired images and capture complementary information through modality translation. However, these models cannot fully exploit the complementary information from different modalities. In this work, we thus present a novel two-step (intra-modality and inter-modality) curriculum disentanglement learning framework to effectively utilize privileged semi-paired images, i.e. limited paired images that are only available in training, for brain tumor …
The brain tumor segmentation on MRI images is a very difficult and important task which is used in surgical and medical planning and assessments. If experts do the segmentation manually with their own medical knowledge, it will be time-consuming. Therefore, researchers propose methods and systems which can do the segmentation automatically and without any interference. In this article, an unsupervised automatic method for brain tumor segmentation on MRI images is presented. In this method, at first in the pre-processing level, the extra parts which are outside the skull and don't have any help…
In this paper, we propose a novel learning based method for automated segmenta-tion of brain tumor in multimodal MRI images. The machine learned features from fully convolutional neural network (FCN) and hand-designed texton fea-tures are used to classify the MRI image voxels. The score map with pixel-wise predictions is used as a feature map which is learned from multimodal MRI train-ing dataset using the FCN. The learned features are then applied to random for-ests to classify each MRI image voxel into normal brain tissues and different parts of tumor. The method was evaluated on BRATS 2013 …
University of Minho（米尼奥大学）；University Hospital Inselspital（因塞尔施皮塔尔大学医院）；University of Bern（伯尔尼大学）
Glioblastoma Multiforme is a high grade, very aggressive, brain tumor, with patients having a poor prognosis. Lower grade gliomas are less aggressive, but they can evolve into higher grade tumors over time. Patient management and treatment can vary considerably with tumor grade, ranging from tumor resection followed by a combined radio- and chemotherapy to a "wait and see" approach. Hence, tumor grading is important for adequate treatment planning and monitoring. The gold standard for tumor grading relies on histopathological diagnosis of biopsy specimens. However, this procedure is invasive, …
Background: Glioma is the most common brain malignant tumor, with a high morbidity rate and a mortality rate of more than three percent, which seriously endangers human health. The main method of acquiring brain tumors in the clinic is MRI. Segmentation of brain tumor regions from multi-modal MRI scan images is helpful for treatment inspection, post-diagnosis monitoring, and effect evaluation of patients. However, the common operation in clinical brain tumor segmentation is still manual segmentation, lead to its time-consuming and large performance difference between different operators, a con…
