Radiomics is an active area of research focusing on high throughput feature extraction from medical images with a wide array of applications in clinical practice, such as clinical decision support in oncology. However, noise in low dose computed tomography (CT) scans can impair the accurate extraction of radiomic features. In this article, we investigate the possibility of using deep learning generative models to improve the performance of radiomics from low dose CTs. We used two datasets of low dose CT scans -NSCLC Radiogenomics and LIDC-IDRI - as test datasets for two tasks - pre-treatment survival prediction and lung cancer diagnosis. We used encoder-decoder networks and conditional generative adversarial networks (CGANs) trained in a previous study as generative models to transform low dose CT images into full dose CT images. Radiomic features extracted from the original and improved CT scans were used to build two classifiers - a support vector machine (SVM) and a deep attention based multiple instance learning model - for survival prediction and lung cancer diagnosis respectively. Finally, we compared the performance of the models derived from the original and improved CT scans. Encoder-decoder networks and CGANs improved the area under the curve (AUC) of survival prediction from 0.52 to 0.57 (p-value<0.01). On the other hand, Encoder-decoder network and CGAN can improve the AUC of lung cancer diagnosis from 0.84 to 0.88 and 0.89 respectively (p-value<0.01). Moreover, there are no statistically significant differences in improving AUC by using encoder-decoder network and CGAN (p-value=0.34) when networks trained at 75 and 100 epochs. Generative models can improve the performance of low dose CT-based radiomics in different tasks. Hence, denoising using generative models seems to be a necessary pre-processing step for calculating radiomic features from low dose CTs.
University of Toronto（多伦多大学）；Lunenfeld-Tanenbaum Research Institute, Sinai Health System（西奈卫生系统卢嫩菲尔德-塔南鲍姆研究所）；McMaster University（麦克马斯特大学）；Hamilton Health Sciences（汉密尔顿健康科学中心）；Sunnybrook Health Sciences Centre（桑尼布鲁克健康科学中心）；Sunnybrook Research Institute（桑尼布鲁克研究所）
As an analytic pipeline for quantitative imaging feature extraction and analysis, radiomics has grown rapidly in the past a few years. Recent studies in radiomics aim to investigate the relationship between tumors imaging features and clinical outcomes. Open source radiomics feature banks enable the extraction and analysis of thousands of predefined features. On the other hand, recent advances in deep learning have shown significant potential in the quantitative medical imaging field, raising the research question of whether predefined radiomics features have predictive information in addition…
Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon, Korea（延世大学电气与计算机工程系）；Department of Radiology and Center for Imaging Science, Samsung Medical Center, Sungkyunkwan University School of Medicine, Suwon, Korea（延世大学医学院放射科与成像科学中心）
The de facto standard of dynamic histogram binning for radiomic feature extraction leads to an elevated sensitivity to fluctuations in annotated regions. This may impact the majority of radiomic studies published recently and contribute to issues regarding poor reproducibility of radiomic-based machine learning that has led to significant efforts for data harmonization; however, we believe the issues highlighted here are comparatively neglected, but often remedied by choosing static binning. The field of radiomics has improved through the development of community standards and open-source libr…
University of British Columbia（不列颠哥伦比亚大学）；BC Cancer Research Institute（不列颠哥伦比亚癌症研究所）；Geneva University Hospital（日内瓦大学医院）
Radiomics features extract quantitative information from medical images, towards the derivation of biomarkers for clinical tasks, such as diagnosis, prognosis, or treatment response assessment. Different image discretization parameters (e.g. bin number or size), convolutional filters, segmentation perturbation, or multi-modality fusion levels can be used to generate radiomics features and ultimately signatures. Commonly, only one set of parameters is used; resulting in only one value or flavour for a given RF. We propose tensor radiomics (TR) where tensors of features calculated with multiple …
Stony Brook University（石溪大学）；Newark Beth Israel Medical Center（纽瓦克贝丝以色列医疗中心）；University Hospitals Cleveland Medical Center（大学医院克利夫兰医疗中心）
We predict mechanical ventilation requirement and mortality using computational modeling of chest radiographs (CXRs) for coronavirus disease 2019 (COVID-19) patients. This two-center, retrospective study analyzed 530 deidentified CXRs from 515 COVID-19 patients treated at Stony Brook University Hospital and Newark Beth Israel Medical Center between March and August 2020. DL and machine learning classifiers to predict mechanical ventilation requirement and mortality were trained and evaluated using patient CXRs. A novel radiomic embedding framework was also explored for outcome prediction. All …
University of Waterloo（滑铁卢大学）；University of Toronto（多伦多大学）；Sunnybrook Health Sciences Centre（桑尼布鲁克健康科学中心）
Prostate cancer is the most diagnosed form of cancer in Canadian men, and is the third leading cause of cancer death. Despite these statistics, prognosis is relatively good with a sufficiently early diagnosis, making fast and reliable prostate cancer detection crucial. As imaging-based prostate cancer screening, such as magnetic resonance imaging (MRI), requires an experienced medical professional to extensively review the data and perform a diagnosis, radiomics-driven methods help streamline the process and has the potential to significantly improve diagnostic accuracy and efficiency, and thu…
Duke Kunshan University（昆山杜克大学）；Duke University（杜克大学）
We developed a deep ensemble learning model with a radiomics spatial encoding execution for improved glioma segmentation accuracy using multi-parametric MRI (mp-MRI). This model was developed using 369 glioma patients with a 4-modality mp-MRI protocol: T1, contrast-enhanced T1 (T1-Ce), T2, and FLAIR. In each modality volume, a 3D sliding kernel was implemented across the brain to capture image heterogeneity: fifty-six radiomic features were extracted within the kernel, resulting in a 4th order tensor. Each radiomic feature can then be encoded as a 3D image volume, namely a radiomic feature map…
University of Waterloo（滑铁卢大学）；Sunnybrook Research Institute（桑尼布鲁克研究所）；University of Toronto（多伦多大学）
Radiomics has proven to be a powerful prognostic tool for cancer detection, and has previously been applied in lung, breast, prostate, and head-and-neck cancer studies with great success. However, these radiomics-driven methods rely on pre-defined, hand-crafted radiomic feature sets that can limit their ability to characterize unique cancer traits. In this study, we introduce a novel discovery radiomics framework where we directly discover custom radiomic features from the wealth of available medical imaging data. In particular, we leverage novel StochasticNet radiomic sequencers for extractin…
Concordia University（康考迪亚大学）；University of Toronto（多伦多大学）；Sunnybrook Health Sciences Centre（桑尼布鲁克健康科学中心）
Recent advancements in signal processing and machine learning coupled with developments of electronic medical record keeping in hospitals and the availability of extensive set of medical images through internal/external communication systems, have resulted in a recent surge of significant interest in "Radiomics". Radiomics is an emerging and relatively new research field, which refers to extracting semi-quantitative and/or quantitative features from medical images with the goal of developing predictive and/or prognostic models, and is expected to become a critical component for integration of …
Precision-medicine-toolbox: An open-source python package for facilitation of quantitative medical imaging and radiomics analysis
The D-Lab, Department of Precision Medicine（精准医学系D实验室）；GROW—School for Oncology, Maastricht University（马斯特里赫特大学GROW肿瘤学院）；GIGA Cyclotrone Research Center（GIGA回旋加速器研究中心）；University of Liege（列日大学）；Department of Radiology and Nuclear Medicine（放射与核医学系）；Maastricht University Medical Centre（马斯特里赫特大学医学中心）
Open-radiomics: A Collection of Standardized Datasets and a Technical Protocol for Reproducible Radiomics Machine Learning Pipelines
The Hospital for Sick Children (SickKids)（SickKids医院（病童医院））；SickKids Research Institute（SickKids研究所）；University of Toronto（多伦多大学）；University Hospital Augsburg（奥格斯堡大学医院）；Institute of Medical Science, University of Toronto（多伦多大学医学科学研究所）；Department of Computer Science, University of Toronto（多伦多大学计算机科学系）；Department of Mechanical and Industrial Engineering, University of Toronto（多伦多大学机械与工业工程系）；Vector Institute（向量研究院）
University Hospital Erlangen（埃尔朗根大学医院）；Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN)（埃尔朗根-EMN综合癌症中心）；Pattern Recognition Lab（模式识别实验室）
Stereotactic radiotherapy (SRT) is one of the most important treatment for patients with brain metastases (BM). Conventionally, following SRT patients are monitored by serial imaging and receive salvage treatments in case of significant tumor growth. We hypothesized that using radiomics and machine learning (ML), metastases at high risk for subsequent progression could be identified during follow-up prior to the onset of significant tumor growth, enabling personalized follow-up intervals and early selection for salvage treatment. All experiments are performed on a dataset from clinical routine…
Fudan University Shanghai Cancer Center（复旦大学附属肿瘤医院）；Shanghai Medical College, Fudan University（复旦大学上海医学院）；Fudan University（复旦大学）；Shanghai Engineering Research Center of Molecular Imaging Probes（上海分子影像探针工程技术研究中心）；Key Laboratory of Nuclear Physics and Ion-beam Application (MOE), Fudan University（复旦大学核物理与离子束应用教育部重点实验室）；the University of Sydney（悉尼大学）；Shanghai Proton and Heavy Ion Center（上海市质子重离子医院）
Objective: Deep Learning-based Radiomics (DLR) has achieved great success in medical image analysis and has been considered a replacement for conventional radiomics that relies on handcrafted features. In this study, we aimed to explore the capability of DLR for the prediction of 5-year Progression-Free Survival (PFS) in Nasopharyngeal Carcinoma (NPC) using pretreatment PET/CT. Methods: A total of 257 patients (170/87 in internal/external cohorts) with advanced NPC (TNM stage III or IVa) were enrolled. We developed an end-to-end multi-modality DLR model, in which a 3D convolutional neural netw…
Technical University of Munich（慕尼黑工业大学）；University of Zurich（苏黎世大学）；Southern University of Science and Technology（南方科技大学）；ETH Zurich（苏黎世联邦理工学院）；Macau University of Science and Technology（澳门科技大学）；Klinikum rechts der Isar（伊萨尔河右岸医院）
Radiomic representations can quantify properties of regions of interest in medical image data. Classically, they account for pre-defined statistics of shape, texture, and other low-level image features. Alternatively, deep learning-based representations are derived from supervised learning but require expensive annotations from experts and often suffer from overfitting and data imbalance issues. In this work, we address the challenge of learning representations of 3D medical images for an effective quantification under data imbalance. We propose a \emph{self-supervised} representation learning…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Weill Cornell Medicine（威尔康奈尔医学院）
Chest X-ray becomes one of the most common medical diagnoses due to its noninvasiveness. The number of chest X-ray images has skyrocketed, but reading chest X-rays still have been manually performed by radiologists, which creates huge burnouts and delays. Traditionally, radiomics, as a subfield of radiology that can extract a large number of quantitative features from medical images, demonstrates its potential to facilitate medical imaging diagnosis before the deep learning era. In this paper, we develop an end-to-end framework, ChexRadiNet, that can utilize the radiomics features to improve t…
Maastricht University Medical Centre+（马斯特里赫特大学医学中心+）；Radboud University Medical Center（拉德堡德大学医学中心）
Radiomics is an active area of research in medical image analysis, the low reproducibility of radiomics has limited its applicability to clinical practice. This issue is especially prominent when radiomic features are calculated from noisy images, such as low dose computed tomography (CT) scans. In this article, we investigate the possibility of improving the reproducibility of radiomic features calculated on noisy CTs by using generative models for denoising.One traditional denoising method - non-local means - and two generative models - encoder-decoder networks (EDN) and conditional generati…
Machine Learning Prediction of Cardiovascular Risk in Type 1 Diabetes Mellitus Using Radiomics Features from Multimodal Retinal Images
Universitat Politècnica de Catalunya (UPC)（加泰罗尼亚理工大学）；Institut Clínic d´Oftalmología (ICOF)（眼科临床研究所）；Hospital Clínic de Barcelona（巴塞罗那临床医院）；Intelligent Data Science and Artificial Intelligence (IDEAI-UPC) Research Center（智能数据科学与人工智能研究中心）；August Pi i Sunyer Biomedical Research Institute (IDIBAPS)（奥古斯特·皮伊·苏涅尔生物医学研究所）；School of Medicine, Universitat de Barcelona（巴塞罗那大学医学院）
An Interpretable Deep Learning Framework for Discovery and Clinical Validation of Deep Radiomic Signatures in Tumor Classification
Department of Health Outcomes and Biomedical Informatics, University of Florida（健康结果与生物医学信息学系，佛罗里达大学）；Tiktok；Department of Biostatistics, MD Anderson Cancer Center（生物统计学系，MD安德森癌症中心）；Department of Urology, University of Florida（泌尿科，佛罗里达大学）
Radiomic Feature Selection Using Gradient Loss of Deep Neural Network for Lung Cancer Stage Detection
Department of Software Engineering, Bahria University（巴尔ia大学软件工程系）；Global College of Engineering and Technology（全球工程与技术学院）；Software Engineering & Business Incubation Center, Bahria University（软件工程与企业孵化中心，巴尔ia大学）
University of Toronto（多伦多大学）；Lunenfeld-Tanenbaum Research Institute（卢嫩菲尔德-塔南鲍姆研究所）；Sinai Health System（西奈健康系统）；McMaster University（麦克马斯特大学）；Hamilton Health Sciences（汉密尔顿健康科学）；Juravinski Hospital and Cancer Centre（朱拉文斯基医院及癌症中心）；Sunnybrook Health Sciences Centre（桑尼布鲁克健康科学中心）；Sunnybrook Research Institute（桑尼布鲁克研究所）
Pancreatic Ductal Adenocarcinoma (PDAC) is one of the most aggressive cancers with an extremely poor prognosis. Radiomics has shown prognostic ability in multiple types of cancer including PDAC. However, the prognostic value of traditional radiomics pipelines, which are based on hand-crafted radiomic features alone is limited. Convolutional neural networks (CNNs) have been shown to outperform these feature-based models in computer vision tasks. However, training a CNN from scratch needs a large sample size which is not feasible in most medical imaging studies. As an alternative solution, CNN-b…
Artificial intelligence (AI) techniques have significant potential to enable effective, robust and automated image phenotyping including identification of subtle patterns. AI-based detection searches the image space to find the regions of interest based on patterns and features. There is a spectrum of tumor histologies from benign to malignant that can be identified by AI-based classification approaches using image features. The extraction of minable information from images gives way to the field of radiomics and can be explored via explicit (handcrafted/engineered) and deep radiomics framewor…
Embedding Radiomics into Vision Transformers for Multimodal Medical Image Classification
Duke Kunshan University（昆山杜克大学）；Jiangsu Provincial University Key (Construction) Laboratory for Smart Diagnosis and Treatment of Lung Cancer（江苏省高校重点（建设）实验室肺癌智能诊疗实验室）；The First People’s Hospital of Kunshan（昆山市第一人民医院）；Duke University（杜克大学）
Anatomy-Informed Deep Learning and Radiomics for Automated Neurofibroma Segmentation in Whole-Body MRI
Institute for Applied Medical Informatics（应用医学信息学研究所）；Institute of Computational Neuroscience（计算神经科学研究所）；Center for Biomedical Artificial Intelligence (bAIome)（生物医学人工智能中心）；University Medical Center Hamburg-Eppendorf（汉堡埃彭多夫大学医学中心）；Department of Diagnostic and Interventional Radiology and Nuclear Medicine（诊断与介入放射学及核医学部门）；Department of Neurology（神经内科部门）
Accurate segmentation of different sub-regions of gliomas including peritumoral edema, necrotic core, enhancing and non-enhancing tumor core from multimodal MRI scans has important clinical relevance in diagnosis, prognosis and treatment of brain tumors. However, due to the highly heterogeneous appearance and shape, segmentation of the sub-regions is very challenging. Recent development using deep learning models has proved its effectiveness in the past several brain segmentation challenges as well as other semantic and medical image segmentation problems. Most models in brain tumor segmentati…
Indian Statistical Institute（印度统计研究所）；University of Calcutta（加尔各答大学）；University of Genova（热那亚大学）
Glioma constitutes 80% of malignant primary brain tumors and is usually classified as HGG and LGG. The LGG tumors are less aggressive, with slower growth rate as compared to HGG, and are responsive to therapy. Tumor biopsy being challenging for brain tumor patients, noninvasive imaging techniques like Magnetic Resonance Imaging (MRI) have been extensively employed in diagnosing brain tumors. Therefore automated systems for the detection and prediction of the grade of tumors based on MRI data becomes necessary for assisting doctors in the framework of augmented intelligence. In this paper, we t…
ARC Training Centre for Innovative BioEngineering（ARC创新生物工程培训中心）
Distant metastases (DM) refer to the dissemination of tumors, usually, beyond the organ where the tumor originated. They are the leading cause of death in patients with soft-tissue sarcomas (STSs). Positron emission tomography-computed tomography (PET-CT) is regarded as the imaging modality of choice for the management of STSs. It is difficult to determine from imaging studies which STS patients will develop metastases. 'Radiomics' refers to the extraction and analysis of quantitative features from medical images and it has been employed to help identify such tumors. The state-of-the-art in ra…
As a means to extract biomarkers from medical imaging, radiomics has attracted increased attention from researchers. However, reproducibility and performance of radiomics in low dose CT scans are still poor, mostly due to noise. Deep learning generative models can be used to denoise these images and in turn improve radiomics' reproducibility and performance. However, most generative models are trained on paired data, which can be difficult or impossible to collect. In this article, we investigate the possibility of denoising low dose CTs using cycle generative adversarial networks (GANs) to im…
Computer aided diagnosis (CAD) of Breast Cancer (BRCA) images has been an active area of research in recent years. The main goals of this research is to develop reliable automatic methods for detecting and diagnosing different types of BRCA from diagnostic images. In this paper, we present a review of the state of the art CAD methods applied to magnetic resonance (MRI) and mammography images of BRCA patients. The review aims to provide an extensive introduction to different features extracted from BRCA images through texture and statistical analysis and to categorize deep learning frameworks a…
Report on the AAPM Grand Challenge on deep generative modeling for learning medical image statistics
Washington University in St. Louis（圣路易斯华盛顿大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Center for Devices and Radiological Health, Food and Drug Administration（美国食品药品监督管理局器械与放射卫生中心）；Puente Solutions, LLC（普恩特解决方案有限责任公司）；Analog Devices, Inc.（亚德诺半导体公司）
Radiomics is a promising technology that focuses on improvements of image analysis, using an automated high-throughput extraction of quantitative features. However, the character of lesion is affected by the surrounding tissue. A lesion on medical image should be characterized from the inter-relation between lesion and surrounding tissue as well as property of the lesion itself. The aim of this study is to introduce a new radiomics feature which quantitatively analyze the inter-relation between lesion and surrounding tissue focusing on the value change of rows and columns in a medical image.
A Novel Patch-Based TDA Approach for Computed Tomography Imaging
School of Computing, Queen’s University（女王大学计算机学院）；The University Of Buckingham（伯克贝克大学）；Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）；Department of Radiology（放射科）；Hepatopancreatobiliary Service, Department of Surgery（肝胆胰胆服务，外科部门）；University of Alberta（阿尔伯塔大学）；Alberta Machine Intelligence Institute（阿尔伯塔机器智能研究所）
Guilin University of Electronic Technology（桂林电子科技大学）；Ecole de technologie Superieure（高等技术学院）
This paper proposes to encode the distribution of features learned from a convolutional neural network using a Gaussian Mixture Model. These parametric features, called GMM-CNN, are derived from chest computed tomography and X-ray scans of patients with Coronavirus Disease 2019. We use the proposed GMM-CNN features as input to a robust classifier based on random forests to differentiate between COVID-19 and other pneumonia cases. Our experiments assess the advantage of GMM-CNN features compared to standard CNN classification on test images. Using a random forest classifier (80\% samples for tr…
Noninvasive H3 K27M screening in pediatric diffuse midline glioma using radiomics on heterogeneous T2-weighted MRI
Innopolis University（因诺波利斯大学）；BRAIn Lab（BRAIn实验室）；Russian Research Center of Roentgenology and Radiology（俄罗斯放射学与放射学研究中心）；Moscow Institute of Physics and Technology（莫斯科物理技术学院）
Tandem mass spectrometry (MS/MS) is a high-throughput technology used toidentify the proteins in a complex biological sample, such as a drop of blood. A collection of spectra is generated at the output of the process, each spectrum of which is representative of a peptide (protein subsequence) present in the original complex sample. In this work, we leverage the log-likelihood gradients of generative models to improve the identification of such spectra. In particular, we show that the gradient of a recently proposed dynamic Bayesian network (DBN) may be naturally employed by a kernel-based disc…
