Weakly Supervised Learning with Automated Labels from Radiology Reports for Glioma Change Detection
Lausanne University Hospital (CHUV)（洛桑大学医院）；University of Lausanne (Unil)（洛桑大学）；CIBM, Center for Biomedical Imaging（生物医学影像中心）；Neuroscience Research Center（神经科学研究中心）；Lundin Brain Tumor Research Center（Lundin脑肿瘤研究中心）
Tommaso Di Noto, Meritxell Bach Cuadra, Chirine Atat, Eduardo Gamito Teiga, Monika Hegi, Andreas Hottinger, Patric Hagmann, Jonas Richiardi
备注：
This work has been submitted as Original Paper to a Journal
Gliomas are the most frequent primary brain tumors in adults. Glioma change detection aims at finding the relevant parts of the image that change over time. Although Deep Learning (DL) shows promising performances in similar change detection tasks, the creation of large annotated datasets represents a major bottleneck for supervised DL applications in radiology. To overcome this, we propose a combined use of weak labels (imprecise, but fast-to-create annotations) and Transfer Learning (TL). Specifically, we explore inductive TL, where source and target domains are identical, but tasks are different due to a label shift: our target labels are created manually by three radiologists, whereas our source weak labels are generated automatically from radiology reports via NLP. We frame knowledge transfer as hyperparameter optimization, thus avoiding heuristic choices that are frequent in related works. We investigate the relationship between model size and TL, comparing a low-capacity VGG with a higher-capacity ResNeXt model. We evaluate our models on 1693 T2-weighted magnetic resonance imaging difference maps created from 183 patients, by classifying them into stable or unstable according to tumor evolution. The weak labels extracted from radiology reports allowed us to increase dataset size more than 3-fold, and improve VGG classification results from 75% to 82% AUC. Mixed training from scratch led to higher performance than fine-tuning or feature extraction. To assess generalizability, we ran inference on an open dataset (BraTS-2015: 15 patients, 51 difference maps), reaching up to 76% AUC. Overall, results suggest that medical imaging problems may benefit from smaller models and different TL strategies with respect to computer vision datasets, and that report-generated weak labels are effective in improving model performances. Code, in-house dataset and BraTS labels are released.
Institute of Cancer Research（癌症研究所）；Universidad de Valladolid（巴利亚多利德大学）；AGH University of Science and Technology（AGH科技大学）；Cardiff University（卡迪夫大学）；Aberystwyth University（阿伯里斯特威斯大学）
Gliomas are brain tumours with a high mortality rate. There are various grades and sub-types of this tumour, and the treatment procedure varies accordingly. Clinicians and oncologists diagnose and categorise these tumours based on visual inspection of radiology and histology data. However, this process can be time-consuming and subjective. The computer-assisted methods can help clinicians to make better and faster decisions. In this paper, we propose a pipeline for automatic classification of gliomas into three sub-types: oligodendroglioma, astrocytoma, and glioblastoma, using both radiology a…
This paper discusses the deep learning architectures currently used for pixel-wise segmentation of primary and secondary glioblastomas and low-grade gliomas. We implement various models such as the popular UNet architecture and compare the performance of these implementations on the BRATS dataset. This paper will explore the different approaches and combinations, offering an in depth discussion of how they perform and how we may improve upon them using more recent advancements in deep learning architectures.
Clinical decision-making in oncology involves multimodal data such as radiology scans, molecular profiling, histopathology slides, and clinical factors. Despite the importance of these modalities individually, no deep learning framework to date has combined them all to predict patient prognosis. Here, we predict the overall survival (OS) of glioma patients from diverse multimodal data with a Deep Orthogonal Fusion (DOF) model. The model learns to combine information from multiparametric MRI exams, biopsy-based modalities (such as H&E slide images and/or DNA sequencing), and clinical variables …
The Hospital for Sick Children（病童医院）；University of Toronto（多伦多大学）；Vector Institute（Vector研究所）
Deep convolutional neural networks (CNNs) have become an essential tool in the medical imaging-based computer-aided diagnostic pipeline. However, training accurate and reliable CNNs requires large fine-grain annotated datasets. To alleviate this, weakly-supervised methods can be used to obtain local information from global labels. This work proposes the use of localized perturbations as a weakly-supervised solution to extract segmentation masks of brain tumours from a pretrained 3D classification model. Furthermore, we propose a novel optimal perturbation method that exploits 3D superpixels to…
SGRNet: Spatially Guided Radiology Network for Structured Radiological Reporting of Head and Neck Cancer
University of Strasbourg（斯特拉斯堡大学）；IHU Strasbourg（斯特拉斯堡大学医院研究所）；Indian Institute of Technology (IIT) Madras（印度马德拉斯理工学院）；All India Institute of Medical Sciences（全印度医学科学院）
Deep Omni-supervised Learning for Rib Fracture Detection from Chest Radiology Images
Imsight AI Research Lab（Imsight AI 研究实验室）；The Hong Kong University of Science and Technology（香港科技大学）；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute（香港科技大学深圳-香港协同创新研究院）；The Chinese University of Hong Kong（香港中文大学）
Johns Hopkins University（约翰斯·霍普金斯大学）；University of Maryland School of Medicine（马里兰大学医学院）
Modern machine learning pipelines, in particular those based on deep learning (DL) models, require large amounts of labeled data. For classification problems, the most common learning paradigm consists of presenting labeled examples during training, thus providing strong supervision on what constitutes positive and negative samples. This constitutes a major obstacle for the development of DL models in radiology--in particular for cross-sectional imaging (e.g., computed tomography [CT] scans)--where labels must come from manual annotations by expert radiologists at the image or slice-level. The…
Towards Generalist Intelligence in Dentistry: Vision Foundation Models for Oral and Maxillofacial Radiology
Shanghai Jiao Tong University（上海交通大学）；Shanghai Ninth People’s Hospital, Shanghai Jiao Tong University School of Medicine, Department of Oral Craniomaxillofacial（上海第九人民医院，上海交通大学医学院，口腔颅面医学部）；Shanghai Jiao Tong University, School of Computer Science（上海交通大学，计算机科学学院）；University of Electronic Science and Technology of China, School of Mechanical and Electrical Engineering（电子科技大学，机械与电子工程学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Shanghai Jiao Tong University, College of Stomatology（上海交通大学，口腔医学院）；National Center for Stomatology（国家口腔医学中心）；National Clinical Medical Research Center for Oral Diseases（国家口腔疾病临床医学研究中心）；Shanghai Key Laboratory of Stomatology（上海口腔医学重点实验室）；Shanghai Research Institute of Stomatology（上海口腔研究所）；Chinese Academy of Medical Science, Research Unit of Oral and Maxillofacial Regenerative Medicine（中国医学科学院，口腔颌面再生医学研究单元）
National Institutes of Health, Clinical Center（美国国立卫生研究院临床中心）；Ping An Technology Co., Ltd.（平安科技有限公司）
In this work, we exploit the task of joint classification and weakly supervised localization of thoracic diseases from chest radiographs, with only image-level disease labels coupled with disease severity-level (DSL) information of a subset. A convolutional neural network (CNN) based attention-guided curriculum learning (AGCL) framework is presented, which leverages the severity-level attributes mined from radiology reports. Images in order of difficulty (grouped by different severity-levels) are fed to CNN to boost the learning gradually. In addition, highly confident samples (measured by cla…
Leveraging Foundation Models for Content-Based Image Retrieval in Radiology
German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing（德国癌症研究中心（DKFZ）海德堡分部医学影像计算部门）；Faculty of Mathematics and Computer Science, Heidelberg University（海德堡大学数学与计算机科学学院）；Medical Faculty Heidelberg, University of Heidelberg（海德堡大学医学学院）；Department of Radiology, Charité - Universitätsmedizin Berlin（柏林夏里特医学院放射科）；German Cancer Research Center (DKFZ) Heidelberg, Interactive Machine Learning Group（德国癌症研究中心（DKFZ）海德堡交互式机器学习小组）
Arizona State University（亚利桑那州立大学）；Barrow Neurological Institute（巴罗神经学研究所）；Irkutsk State Medical University（伊尔库茨克州立医科大学）
Confocal Laser Endomicroscope (CLE) is a novel handheld fluorescence imaging device that has shown promise for rapid intraoperative diagnosis of brain tumor tissue. Currently CLE is capable of image display only and lacks an automatic system to aid the surgeon in analyzing the images. The goal of this project was to develop a computer-aided diagnostic approach for CLE imaging of human glioma with feature localization function. Despite the tremendous progress in object detection and image segmentation methods in recent years, most of such methods require large annotated datasets for training. H…
Tumor Location-weighted MRI-Report Contrastive Learning: A Framework for Improving the Explainability of Pediatric Brain Tumor Diagnosis
The Hospital for Sick Children（病童医院）；University of Toronto（多伦多大学）；Vector Institute for Artificial Intelligence（矢量人工智能研究所）；University Hospital Augsburg（奥格斯堡大学医院）
FDG PET/CT imaging is a resource intensive examination critical for managing malignant disease and is particularly important for longitudinal assessment during therapy. Approaches to automate longtudinal analysis present many challenges including lack of available longitudinal datasets, managing complex large multimodal imaging examinations, and need for detailed annotations for traditional supervised machine learning. In this work we develop OncoNet, novel machine learning algorithm that assesses treatment response from a 1,954 pairs of sequential FDG PET/CT exams through weak supervision usi…
University of Pittsburgh（匹兹堡大学）；University of Pittsburgh Medical Center（匹兹堡大学医学中心）
Creating a large-scale dataset of abnormality annotation on medical images is a labor-intensive and costly task. Leveraging weak supervision from readily available data such as radiology reports can compensate lack of large-scale data for anomaly detection methods. However, most of the current methods only use image-level pathological observations, failing to utilize the relevant anatomy mentions in reports. Furthermore, Natural Language Processing (NLP)-mined weak labels are noisy due to label sparsity and linguistic ambiguity. We propose an Anatomy-Guided chest X-ray Network (AGXNet) to addr…
Weakly-supervised segmentation using inherently-explainable classification models and their application to brain tumour classification
Knowledge Engineering Group, Faculty of Computer Science, Otto von Guericke University Magdeburg, Germany；Biomedical Magnetic Resonance, Faculty of Nature Sciences, Otto von Guericke University Magdeburg, Germany；Genomics Research Centre, Human Technopole, Milan, Italy；Institute for Medical Engineering, Faculty of Electrical Engineering；Information Technology, Otto von Guericke University Magdeburg, Germany；Digital Engineering Faculty, University of Potsdam, Germany；Machine Learning, Hasso-Plattner-Institute, Potsdam, Germany；Department of Biomedical Data Science, Stanford University, Stanford, CA, United States；Centre for Behavioural Brain Sciences, Magdeburg, Germany；German Centre for Neurodegenerative Disease, Magdeburg, Germany
MG-3D: Multi-Grained Knowledge-Enhanced 3D Medical Vision-Language Pre-training
The Hong Kong University of Science and Technology（香港科技大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Shenzhen People’s Hospital（深圳市人民医院）；The University of Hong Kong（香港大学）；Chinese PLA General Hospital（中国人民解放军总医院）
Duke University School of Medicine（杜克大学医学院）；Duke University（杜克大学）；University of Arizona（亚利桑那大学）
Purpose: To design multi-disease classifiers for body CT scans for three different organ systems using automatically extracted labels from radiology text reports.Materials & Methods: This retrospective study included a total of 12,092 patients (mean age 57 +- 18; 6,172 women) for model development and testing (from 2012-2017). Rule-based algorithms were used to extract 19,225 disease labels from 13,667 body CT scans from 12,092 patients. Using a three-dimensional DenseVNet, three organ systems were segmented: lungs and pleura; liver and gallbladder; and kidneys and ureters. For each organ, a t…
Indian Institute of Technology Madras（印度理工学院马德拉斯分校）；Sree Chitra Tirunal Institute for Medical Sciences and Technology（Sree Chitra Tirunal医学科学与技术研究所）
The work presented explores the use of denoising autoencoders (DAE) for brain lesion detection, segmentation and false positive reduction. Stacked denoising autoencoders (SDAE) were pre-trained using a large number of unlabeled patient volumes and fine tuned with patches drawn from a limited number of patients (n=20, 40, 65). The results show negligible loss in performance even when SDAE was fine tuned using 20 patients. Low grade glioma (LGG) segmentation was achieved using a transfer learning approach wherein a network pre-trained with High Grade Glioma (HGG) data was fine tuned using LGG im…
