Latent Correlation Representation Learning for Brain Tumor Segmentation with Missing MRI Modalities
INSA Rouen（鲁昂国立应用科学学院）；Université de Rouen Normandie（鲁昂大学）；Henri Becquerel Cancer Center（亨利·贝克勒尔癌症中心）
Tongxue Zhou, Stéphane Canu, Pierre Vera, Su Ruan
备注：
12 pages, 10 figures, accepted by IEEE Transactions on Image Processing (8 April 2021). arXiv admin note: text overlap with arXiv:2003.08870, arXiv:2102.03111
发表信息：
IEEE Transactions on Image Processing On page(s): 4263-4274 Print ISSN: 1057-7149 Online ISSN: 1941-0042
Magnetic Resonance Imaging (MRI) is a widely used imaging technique to assess brain tumor. Accurately segmenting brain tumor from MR images is the key to clinical diagnostics and treatment planning. In addition, multi-modal MR images can provide complementary information for accurate brain tumor segmentation. However, it's common to miss some imaging modalities in clinical practice. In this paper, we present a novel brain tumor segmentation algorithm with missing modalities. Since it exists a strong correlation between multi-modalities, a correlation model is proposed to specially represent the latent multi-source correlation. Thanks to the obtained correlation representation, the segmentation becomes more robust in the case of missing modality. First, the individual representation produced by each encoder is used to estimate the modality independent parameter. Then, the correlation model transforms all the individual representations to the latent multi-source correlation representations. Finally, the correlation representations across modalities are fused via attention mechanism into a shared representation to emphasize the most important features for segmentation. We evaluate our model on BraTS 2018 and BraTS 2019 dataset, it outperforms the current state-of-the-art methods and produces robust results when one or more modalities are missing.
Vingroup Big Data Institute (VinBigdata)（越南 VinBigdata 大数据研究院）；Hanoi University of Science and Technology (HUST)（河内科技大学）；University of Engineering and Technology (UET), VNU（越南国立大学工程技术大学）
Brain tumor segmentation plays an essential role in medical image analysis. In recent studies, deep convolution neural networks (DCNNs) are extremely powerful to tackle tumor segmentation tasks. We propose in this paper a novel training method that enhances the segmentation results by adding an additional classification branch to the network. The whole network was trained end-to-end on the Multimodal Brain Tumor Segmentation Challenge (BraTS) 2020 training dataset. On the BraTS's validation set, it achieved an average Dice score of 78.43%, 89.99%, and 84.22% respectively for the enhancing tumo…
Treatment-aware Diffusion Probabilistic Model for Longitudinal MRI Generation and Diffuse Glioma Growth Prediction
Oslo University Hospital（奥斯陆大学医院）；Universitat Politècnica de València（瓦伦西亚理工大学）；Instituto Universitario de Tecnologías de la Información y Comunicaciones（信息技术与通信大学学院）
Objective: Magnetic resonance imaging (MRI) has been widely used for the analysis and diagnosis of brain diseases. Accurate and automatic brain tumor segmentation is of paramount importance for radiation treatment. However, low tissue contrast in tumor regions makes it a challenging task.Approach: We propose a novel segmentation network named Category Guided Attention U-Net (CGA U-Net). In this model, we design a Supervised Attention Module (SAM) based on the attention mechanism, which can capture more accurate and stable long-range dependency in feature maps without introducing much computati…
Mohamed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Glioblastoma is a common brain malignancy that tends to occur in older adults and is almost always lethal. The effectiveness of chemotherapy, being the standard treatment for most cancer types, can be improved if a particular genetic sequence in the tumor known as MGMT promoter is methylated. However, to identify the state of the MGMT promoter, the conventional approach is to perform a biopsy for genetic analysis, which is time and effort consuming. A couple of recent publications proposed a connection between the MGMT promoter state and the MRI scans of the tumor and hence suggested the use o…
Indian Statistical Institute（印度统计研究所）；University of Calcutta（加尔各答大学）；University of Genova（热那亚大学）
Glioma constitutes 80% of malignant primary brain tumors and is usually classified as HGG and LGG. The LGG tumors are less aggressive, with slower growth rate as compared to HGG, and are responsive to therapy. Tumor biopsy being challenging for brain tumor patients, noninvasive imaging techniques like Magnetic Resonance Imaging (MRI) have been extensively employed in diagnosing brain tumors. Therefore automated systems for the detection and prediction of the grade of tumors based on MRI data becomes necessary for assisting doctors in the framework of augmented intelligence. In this paper, we t…
University College Dublin（都柏林大学学院）；Université Paris-Est Créteil Val de Marne - Université Paris 12（巴黎东-马恩拉瓦莱大学（巴黎第十二大学））；ADAPT SFI Research Centre（ADAPT SFI研究中心）
Magnetic Resonance Imaging (MRI) is the most commonly used non-intrusive technique for medical image acquisition. Brain tumor segmentation is the process of algorithmically identifying tumors in brain MRI scans. While many approaches have been proposed in the literature for brain tumor segmentation, this paper proposes a lightweight implementation of U-Net. Apart from providing real-time segmentation of MRI scans, the proposed architecture does not need large amount of data to train the proposed lightweight U-Net. Moreover, no additional data augmentation step is required. The lightweight U-Ne…
University of Calcutta（加尔各答大学）；West Bengal University of Technology（西孟加拉技术大学）
Tumor segmentation from magnetic resonance imaging (MRI) data is an important but time consuming manual task performed by medical experts. Automating this process is a challenging task because of the high diversity in the appearance of tumor tissues among different patients and in many cases similarity with the normal tissues. MRI is an advanced medical imaging technique providing rich information about the human soft-tissue anatomy. There are different brain tumor detection and segmentation methods to detect and segment a brain tumor from MRI images. These detection and segmentation approache…
Tumor Location-weighted MRI-Report Contrastive Learning: A Framework for Improving the Explainability of Pediatric Brain Tumor Diagnosis
The Hospital for Sick Children（病童医院）；University of Toronto（多伦多大学）；Vector Institute for Artificial Intelligence（矢量人工智能研究所）；University Hospital Augsburg（奥格斯堡大学医院）
The Segment Anything foundation model achieves favorable brain tumor autosegmentation accuracy on MRI to support radiotherapy treatment planning
Universitätsklinikum Erlangen（埃尔兰根大学医院）；Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希·亚历山大大学）；Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN)（埃尔兰根-欧洲分子神经科学网络综合癌症中心）
Glioma Multimodal MRI Analysis System for Tumor Layered Diagnosis via Multi-task Semi-supervised Learning
Cancer Research Institute, Central South University（中南大学肿瘤研究所）；School of Physics & Electronic Science, Changsha University of Science & Technology（长沙理工大学物理与电子科学学院）；IFLYTEK Research（iFlytek研究院）；School of Life Sciences, Central South University（中南大学生命科学学院）；Department of Radiology, Xiangya Hospital, Central South University（中南大学湘雅医院放射科）；School of Computer Science and Engineering, Central South University（中南大学计算机科学与工程学院）；Department of Neurosurgery, Xiangya Hospital, Central South University（中南大学湘雅医院神经外科）
The Institute of Medical Robotics, Shanghai Jiao Tong University（上海交通大学医疗机器人研究所）；The Hamlyn Centre for Robotic Surgery, Imperial College London（帝国理工学院哈姆林机器人手术中心）；Queen Mary University of London（伦敦玛丽女王大学）；Hangzhou Dianzi University（杭州电子科技大学）
Segmentation of brain tumors and their subregions remains a challenging task due to their weak features and deformable shapes. In this paper, three patterns (cross-skip, skip-1 and skip-2) of distributed dense connections (DDCs) are proposed to enhance feature reuse and propagation of CNNs by constructing tunnels between key layers of the network. For better detecting and segmenting brain tumors from multi-modal 3D MR images, CNN-based models embedded with DDCs (DDU-Nets) are trained efficiently from pixel to pixel with a limited number of parameters. Postprocessing is then applied to refine t…
The brain tumor segmentation on MRI images is a very difficult and important task which is used in surgical and medical planning and assessments. If experts do the segmentation manually with their own medical knowledge, it will be time-consuming. Therefore, researchers propose methods and systems which can do the segmentation automatically and without any interference. In this article, an unsupervised automatic method for brain tumor segmentation on MRI images is presented. In this method, at first in the pre-processing level, the extra parts which are outside the skull and don't have any help…
Comparative Evaluation of Transfer Learning for Classification of Brain Tumor Using MRI
Florida Institute of Technology（佛罗里达理工学院）；Daffodil International University（达福德国际大学）；University of North Texas（北德克萨斯大学）；The University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）
Beijing-Dublin International College（北京-都柏林国际学院）；University College Dublin（都柏林大学）
This technical report presents a comparative analysis of existing deep learning (DL) based approaches for brain tumor segmentation with missing MRI modalities. Approaches evaluated include the Adversarial Co-training Network (ACN) and a combination of mmGAN and DeepMedic. A more stable and easy-to-use version of mmGAN is also open-sourced at a GitHub repository. Using the BraTS2018 dataset, this work demonstrates that the state-of-the-art ACN performs better especially when T1c is missing. While a simple combination of mmGAN and DeepMedic also shows strong potentials when only one MRI modality…
BrainNormalizer: Anatomy-Informed Pseudo-Healthy Brain Reconstruction from Tumor MRI via Edge-Guided ControlNet
University of Pittsburgh（匹兹堡大学）；Georgia Institute of Technology（佐治亚理工学院）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；Cedars-Sinai Medical Center（西德萨斯医疗中心）；Mayo Clinic Arizona（梅奥诊所亚利桑那分部）
University College of Goizha（戈伊扎大学学院）；Sulaimani Polytechnic University（苏莱曼尼理工大学）；Kurdistan Institution for Strategic Studies and Scientific Research（库尔德斯坦战略研究与科学研究所）；University of Kurdistan Hewler（库尔德斯坦休勒大学）
Automated brain tumor detection is becoming a highly considerable medical diagnosis research. In recent medical diagnoses, detection and classification are highly considered to employ machine learning and deep learning techniques. Nevertheless, the accuracy and performance of current models need to be improved for suitable treatments. In this paper, an improvement in deep convolutional learning is ensured by adopting enhanced optimization algorithms, Thus, Deep Convolutional Neural Network (DCNN) based on improved Harris Hawks Optimization (HHO), called G-HHO has been considered. This hybridiz…
Lumbar Spine Tumor Segmentation and Localization in T2 MRI Images Using AI
A. K. Choudhury School of Information Technology, University of Calcutta（加尔各答大学A.K.乔杜里信息技术学院）；Rammohan College, University of Calcutta（加尔各答大学拉莫汉学院）；Indian Statistical Institute（印度统计研究院）
Automatic MRI brain tumor segmentation is of vital importance for the disease diagnosis, monitoring, and treatment planning. In this paper, we propose a two-stage encoder-decoder based model for brain tumor subregional segmentation. Variational autoencoder regularization is utilized in both stages to prevent the overfitting issue. The second-stage network adopts attention gates and is trained additionally using an expanded dataset formed by the first-stage outputs. On the BraTS 2020 validation dataset, the proposed method achieves the mean Dice score of 0.9041, 0.8350, and 0.7958, and Hausdorf…
School of Computer Science, University of Lincoln（林肯大学计算机科学学院）；National Heart & Lung Institute, Imperial College London（帝国理工学院国家心肺研究所）
In this paper, we propose a novel learning based method for automated segmentation of brain tumor in multimodal MRI images, which incorporates two sets of machine -learned and hand crafted features. Fully convolutional networks (FCN) forms the machine learned features and texton based features are considered as hand-crafted features. Random forest (RF) is used to classify the MRI image voxels into normal brain tissues and different parts of tumors, i.e. edema, necrosis and enhancing tumor. The method was evaluated on BRATS 2017 challenge dataset. The results show that the proposed method provi…
Future Processing Healthcare（未来医疗处理公司）；Silesian University of Technology（西里西亚工业大学）；Medical University of Silesia in Katowice（卡托维兹西里西亚医科大学）；ZSM Chorzów（霍茹夫ZSM医院）；University Clinical Centre（大学临床中心）；Roche Pharmaceutical Research & Early Development（罗氏制药研究与早期开发部）
Tumor burden assessment by magnetic resonance imaging (MRI) is central to the evaluation of treatment response for glioblastoma. This assessment is complex to perform and associated with high variability due to the high heterogeneity and complexity of the disease. In this work, we tackle this issue and propose a deep learning pipeline for the fully automated end-to-end analysis of glioblastoma patients. Our approach simultaneously identifies tumor sub-regions, including the enhancing tumor, peritumoral edema and surgical cavity in the first step, and then calculates the volumetric and bidimens…
Massachusetts General Hospital（麻省总医院）；Harvard University（哈佛大学）；General Electric Healthcare（通用电气医疗集团）；Indian Institute of Science（印度科学学院）
Gliomas are one of the most frequent brain tumors and are classified into high grade and low grade gliomas. The segmentation of various regions such as tumor core, enhancing tumor etc. plays an important role in determining severity and prognosis. Here, we have developed a multi-threshold model based on attention U-Net for identification of various regions of the tumor in magnetic resonance imaging (MRI). We propose a multi-path segmentation and built three separate models for the different regions of interest. The proposed model achieved mean Dice Coefficient of 0.59, 0.72, and 0.61 for enhan…
Clinical diagnostic and treatment decisions rely upon the integration of patient-specific data with clinical reasoning. Cancer presents a unique context that influence treatment decisions, given its diverse forms of disease evolution. Biomedical imaging allows noninvasive assessment of disease based on visual evaluations leading to better clinical outcome prediction and therapeutic planning. Early methods of brain cancer characterization predominantly relied upon statistical modeling of neuroimaging data. Driven by the breakthroughs in computer vision, deep learning became the de facto standar…
In this paper, we propose a novel learning based method for automated segmenta-tion of brain tumor in multimodal MRI images. The machine learned features from fully convolutional neural network (FCN) and hand-designed texton fea-tures are used to classify the MRI image voxels. The score map with pixel-wise predictions is used as a feature map which is learned from multimodal MRI train-ing dataset using the FCN. The learned features are then applied to random for-ests to classify each MRI image voxel into normal brain tissues and different parts of tumor. The method was evaluated on BRATS 2013 …
Medical Resonance Imaging or MRI is a medical image processing technique that used radio waves to scan the body. It is a tomographic imaging technique, principally used in the field of radiology. With the advantage of being a painless diagnostic procedure, MRI allows medical personnel to illustrate clear pictures of the anatomy and the physiological processes occurring in the body, thus allowing early detection and treatment of diseases. These images, combined with image processing techniques may be used in the detection of tumors, difficult to identify with the naked eye. This digital assignm…
Shandong Normal University（山东师范大学）；Western University（韦仕敦大学）
Simultaneous segmentation and detection of liver tumors (hemangioma and hepatocellular carcinoma (HCC)) by using multi-modality non-contrast magnetic resonance imaging (NCMRI) are crucial for the clinical diagnosis. However, it is still a challenging task due to: (1) the HCC information on NCMRI is invisible or insufficient makes extraction of liver tumors feature difficult; (2) diverse imaging characteristics in multi-modality NCMRI causes feature fusion and selection difficult; (3) no specific information between hemangioma and HCC on NCMRI cause liver tumors detection difficult. In this stu…
CAVM: Conditional Autoregressive Vision Model for Contrast-Enhanced Brain Tumor MRI Synthesis
School of Medical Technology, Beijing Institute of Technology（北京理工大学医学技术学院）；School of Integrated Circuits and Electronics, Beijing Institute of Technology（北京理工大学集成电路与电子学院）
Department of Computer Science, University of Rochester（罗切斯特大学计算机科学系）；Department of Electrical Engineering, University of Rochester（罗切斯特大学电气工程系）；Department of Biomedical Engineering, University of Rochester（罗切斯特大学生物医学工程系）；Department of Imaging Sciences, University of Rochester（罗切斯特大学影像科学系）；Faculty of Medicine and Institute of Clinical Radiology, Ludwig Maximilian University（路德维希-马克西米利安大学医学院及临床放射学研究所）
Glioma is one of the most common and aggressive types of primary brain tumors. The accurate segmentation of subcortical brain structures is crucial to the study of gliomas in that it helps the monitoring of the progression of gliomas and aids the evaluation of treatment outcomes. However, the large amount of required human labor makes it difficult to obtain the manually segmented Magnetic Resonance Imaging (MRI) data, limiting the use of precise quantitative measurements in the clinical practice. In this work, we try to address this problem by developing a 3D Convolutional Neural Network~(3D C…
From Explanations to Architecture: Explainability-Driven CNN Refinement for Brain Tumor Classification in MRI
American International Brac University Faculty of Psychology University-Bangladesh（美国国际布拉大学心理学学院（孟加拉国））；Shinawatra University Department of Computer Science（辛纳瓦特大学计算机科学系）；American International North South University（美国国际北南大学）；American International University–Bangladesh Department of Computer Science（美国国际大学–孟加拉国计算机科学系）；Multimedia University Department of Computer Science（多媒体大学计算机科学系）
Memorial Sloan Kettering Cancer Center（纪念斯隆·凯特琳癌症中心）
Accurate and robust segmentation of lung cancers from CT, even those located close to mediastinum, is needed to more accurately plan and deliver radiotherapy and to measure treatment response. Therefore, we developed a new cross-modality educed distillation (CMEDL) approach, using unpaired CT and MRI scans, whereby an informative teacher MRI network guides a student CT network to extract features that signal the difference between foreground and background. Our contribution eliminates two requirements of distillation methods: (i) paired image sets by using an image to image (I2I) translation a…
Pakistan Institute of Engineering and Applied Sciences（巴基斯坦工程与应用科学学院）
Brain tumors analysis is important in timely diagnosis and effective treatment to cure patients. Tumor analysis is challenging because of tumor morphology like size, location, texture, and heteromorphic appearance in the medical images. In this regard, a novel two-phase deep learning-based framework is proposed to detect and categorize brain tumors in magnetic resonance images (MRIs). In the first phase, a novel deep boosted features and ensemble classifiers (DBF-EC) scheme is proposed to detect tumor MRI images from healthy individuals effectively. The deep boosted feature space is achieved t…
Pakistan Institute of Engineering and Applied Sciences (PIEAS)（巴基斯坦工程与应用科学学院）；Ibadat International University（伊巴达特国际大学）；University of Engineering and Applied Science (UEAS)（工程与应用科学大学）
Brain tumor classification is crucial for clinical analysis and an effective treatment plan to cure patients. Deep learning models help radiologists to accurately and efficiently analyze tumors without manual intervention. However, brain tumor analysis is challenging because of its complex structure, texture, size, location, and appearance. Therefore, a novel deep residual and regional-based Res-BRNet Convolutional Neural Network (CNN) is developed for effective brain tumor (Magnetic Resonance Imaging) MRI classification. The developed Res-BRNet employed Regional and boundary-based operations …
One of the most important tasks in medical image processing is the brain's whole tumor segmentation. It assists in quicker clinical assessment and early detection of brain tumors, which is crucial for lifesaving treatment procedures of patients. Because, brain tumors often can be malignant or benign, if they are detected at an early stage. A brain tumor is a collection or a mass of abnormal cells in the brain. The human skull encloses the brain very rigidly and any growth inside this restricted place can cause severe health issues. The detection of brain tumors requires careful and intricate a…
Polytechnique Montreal（蒙特利尔理工学院）；Mila, Quebec AI Institute（米拉魁北克人工智能研究所）；Beijing Tiantan Hospital, Capital Medical University（首都医科大学附属北京天坛医院）；Université de Montréal（蒙特利尔大学）
Spinal cord tumors lead to neurological morbidity and mortality. Being able to obtain morphometric quantification (size, location, growth rate) of the tumor, edema, and cavity can result in improved monitoring and treatment planning. Such quantification requires the segmentation of these structures into three separate classes. However, manual segmentation of 3-dimensional structures is time-consuming and tedious, motivating the development of automated methods. Here, we tailor a model adapted to the spinal cord tumor segmentation task. Data were obtained from 343 patients using gadolinium-enha…
Brain Tumor Classification in MRI Images: A Computationally Efficient Convolutional Neural Network
Department of Computer Science and Engineering, University of North Texas, USA（北卡罗来纳州立大学计算机科学与工程系）；Department of Electrical and Electronic Engineering, International Islamic University Chittagong, Bangladesh（伊斯兰国际大学查塔格昂分校电子与电气工程系）
Convolutional neural networks (CNNs) have achieved remarkable success in automatically segmenting organs or lesions on 3D medical images. Recently, vision transformer networks have exhibited exceptional performance in 2D image classification tasks. Compared with CNNs, transformer networks have an appealing advantage of extracting long-range features due to their self-attention algorithm. Therefore, we propose a CNN-Transformer combined model, called BiTr-Unet, with specific modifications for brain tumor segmentation on multi-modal MRI scans. Our BiTr-Unet achieves good performance on the BraTS…
Segmentation of images holds an important position in the area of image processing. It becomes more important whi le typically dealing with medical images where presurgery and post surgery decisions are required for the purpose of initiating and speeding up the recovery process. Segmentation of 3-D tumor structures from magnetic resonance images (MRI) is a very challenging problem due to the variability of tumor geometry and intensity patterns. Level set evolution combining global smoothness with the flexibility of topology changes offers significant advantages over the conventional statistica…
Ashford and St Peter’s Hospitals NHS Foundation Trust（阿什福德和圣彼得医院 NHS 基金会信托）；University of Lincoln（林肯大学）
In this paper we propose a 2D deep residual Unet with 104 convolutional layers (DR-Unet104) for lesion segmentation in brain MRIs. We make multiple additions to the Unet architecture, including adding the 'bottleneck' residual block to the Unet encoder and adding dropout after each convolution block stack. We verified the effect of introducing the regularisation of dropout with small rate (e.g. 0.2) on the architecture, and found a dropout of 0.2 improved the overall performance compared to no dropout, or a dropout of 0.5. We evaluated the proposed architecture as part of the Multimodal Brain …
