When is 3D Worth It? A Resource-Performance Frontier for CNNs and Transformers in Lung CT
Department of Biomedical Informatics University of Arkansas for Medical Sciences（生物医学信息学系，美国阿肯色大学医学科学分校）；Department of Information Science University of Arkansas at Little Rock（信息科学系，美国阿肯色大学小岩分校）；Department of Neuroscience University of Arkansas for Medical Sciences（神经科学系，美国阿肯色大学医学科学分校）
Md Enamul Hoq, Sharafat Hossain, Imraul Emmaka, Linda Larson-Prior, Lawrence Tarbox, Jonathan Bona, Donald Johann Jr. and Fred Prior
三维模型通常被认为更适合体积医学成像，但其实际价值取决于性能提升是否值得增加的计算成本和复杂性。我们不提出新架构，而是研究在固定训练协议下，输入维度（2D、2.5D、3D）如何影响卷积神经网络（CNN）和视觉Transformer（ViT）的行为。使用无泄漏的NLST队列（n=1,977）和辅助LIDC-IDRI数据，我们发现2.5D CNN在我们的比较中提供了最有利的判别-稳定性权衡（ROC-AUC 0.682，95% CI [0.546, 0.799]），具有稳定的操作点。相比之下，3D CNN表现出阈值不稳定性，而Transformer出现退化预测，例如全正预测。置信区间宽且重叠，因此我们将这些结果呈现为受控的资源-性能前沿和失败模式分类，而非明确的优越性声明。对于类别不平衡的肺癌筛查分类，2D和2.5D输入在性能、稳定性和计算效率之间提供了比全3D表示更可靠的权衡。
英文摘要：
Three-dimensional models are widely assumed preferable for volumetric medical imaging, yet their practical value depends on whether performance gains justify added computational cost and complexity. Rather than proposing a new architecture, we study how input dimensionality (2D, 2.5D, 3D) affects model behavior across convolutional neural networks (CNNs) and Vision Transformers (ViTs) under a fixed training protocol. Using a leakage-free NLST cohort (n = 1,977) with supporting LIDC-IDRI data, we find that the 2.5D CNN offers the most favorable discrimination-stability trade-off in our comparison (ROC-AUC 0.682, 95% CI [0.546, 0.799]) with a stable operating point. In contrast, 3D CNNs show threshold instability, and transformers exhibit degenerate predictions, such as all-positive predictions. Confidence intervals are wide and overlapping, so we present these results as a controlled resource-performance frontier and a failure-mode taxonomy rather than as definitive superiority claims. For class-imbalanced lung cancer screening classification, 2D and 2.5D inputs provide a more reliable trade-off between performance, stability, and computational efficiency than full 3D representations.
Recent studies have shown that lung cancer screening using annual low-dose computed tomography (CT) reduces lung cancer mortality by 20% compared to traditional chest radiography. Therefore, CT lung screening has started to be used widely all across the world. However, analyzing these images is a serious burden for radiologists. In this study, we propose a novel and simple framework that analyzes CT lung screenings using convolutional neural networks (CNNs) and reduces false positives. Our framework shows that even non-complex architectures are very powerful to classify 3D nodule data when com…
Deformable registration continues to be one of the key challenges in medical image analysis. While iconic registration methods have started to benefit from the recent advances in medical deep learning, the same does not yet apply for the registration of point sets, e.g. registration based on surfaces, keypoints or landmarks. This is mainly due to the restriction of the convolution operator in modern CNNs to densely gridded input. However, with the newly developed methods from the field of geometric deep learning suitable tools are now emerging, which enable powerful analysis of medical data on…
Traditional survival models such as the Cox proportional hazards model are typically based on scalar or categorical clinical features. With the advent of increasingly large image datasets, it has become feasible to incorporate quantitative image features into survival prediction. So far, this kind of analysis is mostly based on radiomics features, i.e. a fixed set of features that is mathematically defined a priori. To capture highly abstract information, it is desirable to learn the feature extraction using convolutional neural networks. However, for tomographic medical images, model training…
Universitat Rovira i Virgili（罗维拉-威尔吉利大学）；University of Cagliari（卡利亚里大学）
In this paper, we examine the strength of deep learning technique for diagnosing lung cancer on medical image analysis problem. Convolutional neural networks (CNNs) models become popular among the pattern recognition and computer vision research area because of their promising outcome on generating high-level image representations. We propose a new deep learning architecture for learning high-level image representation to achieve high classification accuracy with low variance in medical image binary classification tasks. We aim to learn discriminant compact features at beginning of our deep co…
MSAD-Net: Multiscale and Spatial Attention-based Dense Network for Lung Cancer Classification
Computer Science and Engineering Department, NIIT University, Jaipur, India（尼特大学计算机科学与工程系，印度斋普尔）；CSE department, Christ (Deemed to be University), Bangalore, India（基督大学（认定为大学）计算机科学与工程系，印度班加罗尔）；Faculty of Computing, Engineering and the Built Environment, Birmingham City University, UK（伯明翰城市大学计算、工程与建筑环境学院，英国）
Jordan University of Science and Technology（约旦科技大学）
Purpose: The lung nodules localization in CT scan images is the most difficult task due to the complexity of the arbitrariness of shape, size, and texture of lung nodules. This is a challenge to be faced when coming to developing different solutions to improve detection systems. the deep learning approach showed promising results by using convolutional neural network (CNN), especially for image recognition and it's one of the most used algorithm in computer vision. Approach: we use (CNN) building blocks based on YOLOv5 (you only look once) to learn the features representations for nodule detec…
Chest radiography is one of the most common types of diagnostic radiology exams, which is critical for screening and diagnosis of many different thoracic diseases. Specialized algorithms have been developed to detect several specific pathologies such as lung nodule or lung cancer. However, accurately detecting the presence of multiple diseases from chest X-rays (CXRs) is still a challenging task. This paper presents a supervised multi-label classification framework based on deep convolutional neural networks (CNNs) for predicting the risk of 14 common thoracic diseases. We tackle this problem …
University of Bern（伯尔尼大学）；Bern University Hospital “Inselspital”（伯尔尼大学医院“Inselspital”）
Early diagnosis of interstitial lung diseases is crucial for their treatment, but even experienced physicians find it difficult, as their clinical manifestations are similar. In order to assist with the diagnosis, computer-aided diagnosis (CAD) systems have been developed. These commonly rely on a fixed scale classifier that scans CT images, recognizes textural lung patterns and generates a map of pathologies. In a previous study, we proposed a method for classifying lung tissue patterns using a deep convolutional neural network (CNN), with an architecture designed for the specific problem. In…
Pathology slides of lung malignancies are classified using resource-frugal convolution neural networks (CNNs) that may be deployed on mobile devices. In particular, the challenging task of distinguishing adenocarcinoma (LUAD) and squamous-cell carcinoma (LUSC) lung cancer subtypes is approached in two stages. First, whole-slide histopathology images are downsampled to a size too large for CNN analysis but large enough to retain key anatomic detail. The downsampled images are decomposed into smaller square tiles, which are sifted based on their image entropies. A lightweight CNN produces tile-l…
School of Professional Studies（专业研究学院）；Northwestern University（西北大学）
Artificial intelligence (AI) is disrupting the medical field as advances in modern technology allow common household computers to learn anatomical and pathological features that distinguish between healthy and disease with the accuracy of highly specialized, trained physicians. Computer vision AI applications use medical imaging, such as lung chest X-Rays (LCXRs), to facilitate diagnoses by providing second-opinions in addition to a physician's or radiologist's interpretation. Considering the advent of the current Coronavirus disease (COVID-19) pandemic, LCXRs may provide rapid insights to ind…
School of Biomedical Engineering and Imaging Sciences, King’s College London（伦敦国王学院生物医学工程与影像科学学院）；BioMedIA, Department of Computing, Imperial College London（帝国理工学院计算系生物医学影像与数据分析中心）；Department of Radiology, University College London（伦敦大学学院放射学系）；The Royal Brompton & Harefield NHS Foundation Trust（皇家布朗普顿和哈菲尔德英国国民保健信托基金会）；Technical University of Munich（慕尼黑工业大学）；Helmholtz Center Munich（亥姆霍兹慕尼黑中心）
Convolutional Neural Networks (CNNs) are widely used for image classification in a variety of fields, including medical imaging. While most studies deploy cross-entropy as the loss function in such tasks, a growing number of approaches have turned to a family of contrastive learning-based losses. Even though performance metrics such as accuracy, sensitivity and specificity are regularly used for the evaluation of CNN classifiers, the features that these classifiers actually learn are rarely identified and their effect on the classification performance on out-of-distribution test samples is ins…
Chest radiographs are used for the diagnosis of multiple critical illnesses (e.g., Pneumonia, heart failure, lung cancer), for this reason, systems for the automatic or semi-automatic analysis of these data are of particular interest. An efficient analysis of large amounts of chest radiographs can aid physicians and radiologists, ultimately allowing for better medical care of lung-, heart- and chest-related conditions. We propose a novel Discrete Wavelet Transform (DWT)-based method for the efficient identification and encoding of visual information that is typically lost in the down-sampling …
Lung cancer is the leading cause of cancer-related deaths in the past several years. A major challenge in lung cancer screening is the detection of lung nodules from computed tomography (CT) scans. State-of-the-art approaches in automated lung nodule classification use deep convolutional neural networks (CNNs). However, these networks require a large number of training samples to generalize well. This paper investigates the use of capsule networks (CapsNets) as an alternative to CNNs. We show that CapsNets significantly outperforms CNNs when the number of training samples is small. To increase…
University of Patras（帕特雷大学）；School of Medicine（医学院）；Laboratory of Nuclear Medicine（核医学实验室）
Lung Cancer is the most common cause of cancer-related death worldwide. Early and automatic diagnosis of Solitary Pulmonary Nodules (SPN) in Computer Tomography (CT) chest scans can provide early treatment as well as doctor liberation from time-consuming procedures. Deep Learning has been proven as a popular and influential method in many medical imaging diagnosis areas. In this study, we consider the problem of diagnostic classification between benign and malignant lung nodules in CT images derived from a PET/CT scanner. More specifically, we intend to develop experimental Convolutional Neura…
Monash University Malaysia（莫纳什大学马来西亚分校）；Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局）；CNRS（法国国家科学研究中心）；Singapore Eye Research Institute（新加坡眼科研究所）；University of Oklahoma（俄克拉荷马大学）
Different types of Convolutional Neural Networks (CNNs) have been applied to detect cancerous lung nodules from computed tomography (CT) scans. However, the size of a nodule is very diverse and can range anywhere between 3 and 30 millimeters. The high variation of nodule sizes makes classifying them a difficult and challenging task. In this study, we propose a novel CNN architecture called Gated-Dilated (GD) networks to classify nodules as malignant or benign. Unlike previous studies, the GD network uses multiple dilated convolutions instead of max-poolings to capture the scale variations. Mor…
Harvard University（哈佛大学）；University of Central Florida（中佛罗里达大学）
Lung cancer is the leading cause of cancer related mortality by a significant margin. While new technologies, such as image segmentation, have been paramount to improved detection and earlier diagnoses, there are still significant challenges in treating the disease. In particular, despite an increased number of curative resections, many postoperative patients still develop recurrent lesions. Consequently, there is a significant need for prognostic tools that can more accurately predict a patient's risk for recurrence. In this paper, we explore the use of convolutional neural networks (CNNs) fo…
Classification based deep learning models for lung cancer and disease using medical images
Laboratory for Artificial Intelligence of Personalized Medicine, School of Artificial Intelligence, Guilin University of Electronic Technology（人工智能个性化医学实验室，人工智能学院，桂林电子科技大学）；The Laboratory for Imagery, Vision and Artificial Intelligence, Ecole de Technologie Superieure（影像、视觉与人工智能实验室，蒙特利尔理工学院）
Concordia Institute for Information Systems Engineering, Concordia University（康考迪亚大学信息系统工程研究所）；Sunnybrook Health Sciences Centre, University of Toronto（多伦多大学桑尼布鲁克健康科学中心）；McGill University Health Center- Research Institute（麦吉尔大学健康中心研究所）；Babak Imaging Center（巴布克影像中心）；Department of Electrical and Computer Engineering, University of Toronto（多伦多大学电气与计算机工程系）
Lung diseases including infections such as Pneumonia, Tuberculosis, and novel Coronavirus (COVID-19), together with Lung Cancer are significantly widespread and are, typically, considered life threatening. In particular, lung cancer is among the most common and deadliest cancers with a low 5-year survival rate. Timely diagnosis of lung cancer is, therefore, of paramount importance as it can save countless lives. In this regard, deep learning radiomics solutions have the promise of extracting the most useful features on their own in an end-to-end fashion without having access to the annotated b…
Medical Imaging Center, Vingroup Big Data Institute（Vingroup大数据研究院医学影像中心）；College of Engineering & Computer Science, VinUniversity（VinUniversity工程与计算机科学学院）
Chest radiograph (CXR) interpretation in pediatric patients is error-prone and requires a high level of understanding of radiologic expertise. Recently, deep convolutional neural networks (D-CNNs) have shown remarkable performance in interpreting CXR in adults. However, there is a lack of evidence indicating that D-CNNs can recognize accurately multiple lung pathologies from pediatric CXR scans. In particular, the development of diagnostic models for the detection of pediatric chest diseases faces significant challenges such as (i) lack of physician-annotated datasets and (ii) class imbalance …
COMSATS University Islamabad（伊斯兰堡COMSATS大学）；Bacha Khan University（巴查汗大学）；Chung-Ang University（中央大学）；University of Malaya（马来亚大学）；University Malaysia Sabah（马来西亚沙巴大学）；Institute of Space Technology（空间技术研究所）
The ongoing COVID-19 pandemic has already taken millions of lives and damaged economies across the globe. Most COVID-19 deaths and economic losses are reported from densely crowded cities. It is comprehensible that the effective control and prevention of epidemic/pandemic infectious diseases is vital. According to WHO, testing and diagnosis is the best strategy to control pandemics. Scientists worldwide are attempting to develop various innovative and cost-efficient methods to speed up the testing process. This paper comprehensively evaluates the applicability of the recent top ten state-of-th…
This review presents a comprehensive study on the role of GANs in addressing the challenges related to COVID-19 data scarcity and diagnosis. It is the first review that summarizes the different GANs methods and the lungs images datasets for COVID-19. It attempts to answer the questions related to applications of GANs, popular GAN architectures, frequently used image modalities, and the availability of source code. This review included 57 full-text studies that reported the use of GANs for different applications in COVID-19 lungs images data. Most of the studies (n=42) used GANs for data augmen…
2.75D: Boosting learning by representing 3D Medical imaging to 2D features for small data
Netherlands Cancer Institute（荷兰癌症研究所）；Erasmus Medical Center Rotterdam（鹿特丹伊拉斯姆斯医学中心）；Radboud University Medical Center（拉德堡德大学医学中心）；Southern University of Science and Technology（南方科技大学）；Aberystwyth University（阿伯里斯特威斯大学）
Recent studies have shown that lung cancer screening using annual low-dose computed tomography (CT) reduces lung cancer mortality by 20% compared to traditional chest radiography. Therefore, CT lung screening has started to be used widely all across the world. However, analyzing these images is a serious burden for radiologists. The number of slices in a CT scan can be up to 600. Therefore, computer-aided-detection (CAD) systems are very important for faster and more accurate assessment of the data. In this study, we proposed a framework that analyzes CT lung screenings using convolutional neu…
University of Maryland Baltimore County（马里兰大学巴尔的摩县分校）
We present a hybrid algorithm to estimate lung nodule malignancy that combines imaging biomarkers from Radiologist's annotation with image classification of CT scans. Our algorithm employs a 3D Convolutional Neural Network (CNN) as well as a Random Forest in order to combine CT imagery with biomarker annotation and volumetric radiomic features. We analyze and compare the performance of the algorithm using only imagery, only biomarkers, combined imagery + biomarkers, combined imagery + volumetric radiomic features and finally the combination of imagery + biomarkers + volumetric features in orde…
University of Dhaka（达卡大学）；Qatar University（卡塔尔大学）；Bangabandhu Sheikh Mujib Medical University（邦班杜·谢赫·穆吉布医科大学）；Hamad General Hospital（哈马德综合医院）；Weill Cornell Medicine – Qatar（威尔康奈尔医学院卡塔尔分校）；Universiti Kebangsaan Malaysia（马来西亚国民大学）；North South University（北南大学）；College of Engineering, Qatar University（卡塔尔大学工程学院）
Tuberculosis (TB) is a chronic lung disease that occurs due to bacterial infection and is one of the top 10 leading causes of death. Accurate and early detection of TB is very important, otherwise, it could be life-threatening. In this work, we have detected TB reliably from the chest X-ray images using image pre-processing, data augmentation, image segmentation, and deep-learning classification techniques. Several public databases were used to create a database of 700 TB infected and 3500 normal chest X-ray images for this study. Nine different deep CNNs (ResNet18, ResNet50, ResNet101, ChexNe…
Columbia University（哥伦比亚大学）；Imperial College（帝国理工学院）
Automated detection and segmentation of pulmonary nodules on lung computed tomography (CT) scans can facilitate early lung cancer diagnosis. Existing supervised approaches for automated nodule segmentation on CT scans require voxel-based annotations for training, which are labor- and time-consuming to obtain. In this work, we propose a weakly-supervised method that generates accurate voxel-level nodule segmentation trained with image-level labels only. By adapting a convolutional neural network (CNN) trained for image classification, our proposed method learns discriminative regions from the a…
University of Engineering and Applied Science（工程与应用科学大学）
In December 2019, the global pandemic COVID-19 in Wuhan, China, affected human life and the worldwide economy. Therefore, an efficient diagnostic system is required to control its spread. However, the automatic diagnostic system poses challenges with a limited amount of labeled data, minor contrast variation, and high structural similarity between infection and background. In this regard, a new two-phase deep convolutional neural network (CNN) based diagnostic system is proposed to detect minute irregularities and analyze COVID-19 infection. In the first phase, a novel SB-STM-BRNet CNN is deve…
Rensselaer Polytechnic Institute（伦斯勒理工学院）；Massachusetts General Hospital（麻省总医院）
Recent studies have highlighted the high correlation between cardiovascular diseases (CVD) and lung cancer, and both are associated with significant morbidity and mortality. Low-Dose CT (LCDT) scans have led to significant improvements in the accuracy of lung cancer diagnosis and thus the reduction of cancer deaths. However, the high correlation between lung cancer and CVD has not been well explored for mortality prediction. This paper introduces a knowledge-based analytical method using deep convolutional neural network (CNN) for all-cause mortality prediction. The underlying approach combine…
LHU-Net: a Lean Hybrid U-Net for Cost-efficient, High-performance Volumetric Segmentation
Faculty of Informatics and Data Science, University of Regensburg（信息学与数据科学学院，莱茵河畔罗特魏尔大学）；Faculty of Electrical Engineering and Information Technology, RWTH Aachen University（电气工程与信息科技学院，亚琛工业大学）；Fraunhofer Institute for Digital Medicine MEVIS（弗劳恩霍夫数字医学研究所MEVIS）
While the Transformer architecture has become the de-facto standard for natural language processing tasks, its applications to computer vision remain limited. In vision, attention is either applied in conjunction with convolutional networks, or used to replace certain components of convolutional networks while keeping their overall structure in place. We show that this reliance on CNNs is not necessary and a pure transformer applied directly to sequences of image patches can perform very well on image classification tasks. When pre-trained on large amounts of data and transferred to multiple m…
Is an Image Also Worth 16x16=256 Superpixels? A Framework for Attentional Image Classification
UFRGS（联邦大学里约格兰德杜斯鲁斯）；Institute of Informatics（信息学院）；Federal University of Rio Grande do Sul（里约格兰德杜斯鲁斯联邦大学）；Division of Informatics（信息系）；School of Health Sciences（健康科学学院）；Imaging and Data Science（成像与数据科学）；Faculty of Biology, Medicine and Health（生物医学与健康学院）；University of Manchester（曼彻斯特大学）；Vaughan House, Portsmouth St（波特兰街瓦尔赫恩大楼）
