Medical Knowledge-Guided Deep Learning for Imbalanced Medical Image Classification
University of Pittsburgh（匹兹堡大学）；National University of Defense Technology（国防科技大学）；University of Pittsburgh Medical Center Children’s Hospital of Pittsburgh（匹兹堡大学医学中心匹兹堡儿童医院）；Magee-Womens Hospital of University of Pittsburgh Medical Center（匹兹堡大学医学中心马吉妇女医院）
Long Gao, Chang Liu, Dooman Arefan, Ashok Panigrahy, Margarita L. Zuley, Shandong Wu
备注：
Corrected inaccurate information in affiliation and acknowledgment
Deep learning models have gained remarkable performance on a variety of image classification tasks. However, many models suffer from limited performance in clinical or medical settings when data are imbalanced. To address this challenge, we propose a medical-knowledge-guided one-class classification approach that leverages domain-specific knowledge of classification tasks to boost the model's performance. The rationale behind our approach is that some existing prior medical knowledge can be incorporated into data-driven deep learning to facilitate model learning. We design a deep learning-based one-class classification pipeline for imbalanced image classification, and demonstrate in three use cases how we take advantage of medical knowledge of each specific classification task by generating additional middle classes to achieve higher classification performances. We evaluate our approach on three different clinical image classification tasks (a total of 8459 images) and show superior model performance when compared to six state-of-the-art methods. All codes of this work will be publicly available upon acceptance of the paper.
We propose performing imbalanced classification by regrouping majority classes into small classes so that we turn the problem into balanced multiclass classification. This new idea is dramatically different from popular loss reweighting and class resampling methods. Our preliminary result on imbalanced medical image classification shows that this natural idea can substantially boost the classification performance as measured by average precision (approximately area-under-the-precision-recall-curve, or AUPRC), which is more appropriate for evaluating imbalanced classification than other metrics…
Kyushu University（九州大学）；Kyoto University（京都大学）；The University of Tokyo（东京大学）
Semi-supervised domain adaptation is a technique to build a classifier for a target domain by modifying a classifier in another (source) domain using many unlabeled samples and a small number of labeled samples from the target domain. In this paper, we develop a semi-supervised domain adaptation method, which has robustness to class-imbalanced situations, which are common in medical image classification tasks. For robustness, we propose a weakly-supervised clustering pipeline to obtain high-purity clusters and utilize the clusters in representation learning for domain adaptation. The proposed …
University of Texas at Dallas（德克萨斯大学达拉斯分校）；Indian Institute of Technology, Bombay（印度理工学院孟买分校）
Training deep learning models on medical datasets that perform well for all classes is a challenging task. It is often the case that a suboptimal performance is obtained on some classes due to the natural class imbalance issue that comes with medical data. An effective way to tackle this problem is by using targeted active learning, where we iteratively add data points to the training data that belong to the rare classes. However, existing active learning methods are ineffective in targeting rare classes in medical datasets. In this work, we propose Clinical (targeted aCtive Learning for Imbal…
Medical image classification is often challenging for two reasons: a lack of labelled examples due to expensive and time-consuming annotation protocols, and imbalanced class labels due to the relative scarcity of disease-positive individuals in the wider population. Semi-supervised learning (SSL) methods exist for dealing with a lack of labels, but they generally do not address the problem of class imbalance. In this study we propose Adaptive Blended Consistency Loss (ABCL), a drop-in replacement for consistency loss in perturbation-based SSL methods. ABCL counteracts data skew by adaptively m…
University of Cambridge（剑桥大学）；University of Salerno（萨莱诺大学）
Automatic segmentation methods are an important advancement in medical image analysis. Machine learning techniques, and deep neural networks in particular, are the state-of-the-art for most medical image segmentation tasks. Issues with class imbalance pose a significant challenge in medical datasets, with lesions often occupying a considerably smaller volume relative to the background. Loss functions used in the training of deep learning algorithms differ in their robustness to class imbalance, with direct consequences for model convergence. The most commonly used loss functions for segmentati…
We propose a highly data-efficient active learning framework for image classification. Our novel framework combines: (1) unsupervised representation learning of a Convolutional Neural Network and (2) the Gaussian Process (GP) method, in sequence to achieve highly data and label efficient classifications. Moreover, both elements are less sensitive to the prevalent and challenging class imbalance issue, thanks to the (1) feature learned without labels and (2) the Bayesian nature of GP. The GP-provided uncertainty estimates enable active learning by ranking samples based on the uncertainty and se…
Data imbalance between common and rare diseases during model training often causes intelligent diagnosis systems to have biased predictions towards common diseases. The state-of-the-art approaches apply a two-stage learning framework to alleviate the class-imbalance issue, where the first stage focuses on training of a general feature extractor and the second stage focuses on fine-tuning the classifier head for class rebalancing. However, existing two-stage approaches do not consider the fine-grained property between different diseases, often causing the first stage less effective for medical …
National Library of Medicine, National Institutes of Health（美国国立卫生研究院国家医学图书馆）；Stanford University（斯坦福大学）
In medical image classification tasks, it is common to find that the number of normal samples far exceeds the number of abnormal samples. In such class-imbalanced situations, reliable training of deep neural networks continues to be a major challenge. Under these circumstances, the predicted class probabilities may be biased toward the majority class. Calibration has been suggested to alleviate some of these effects. However, there is insufficient analysis explaining when and whether calibrating a model would be beneficial in improving performance. In this study, we perform a systematic analys…
University of Pittsburgh（匹兹堡大学）；National University of Defense Technology（国防科技大学）；University of Pittsburgh Medical Center Children’s Hospital of Pittsburgh（匹兹堡大学医学中心匹兹堡儿童医院）
Medical image data are usually imbalanced across different classes. One-class classification has attracted increasing attention to address the data imbalance problem by distinguishing the samples of the minority class from the majority class. Previous methods generally aim to either learn a new feature space to map training samples together or to fit training samples by autoencoder-like models. These methods mainly focus on capturing either compact or descriptive features, where the information of the samples of a given one class is not sufficiently utilized. In this paper, we propose a novel …
Bournemouth University（伯恩茅斯大学）；University of Adelaide（阿德莱德大学）；Universitat Pompeu Fabra（庞培法布拉大学）；Catalan Institution for Research and Advanced Studies (ICREA)（加泰罗尼亚研究与高级研究机构（ICREA））
Highly imbalanced datasets are ubiquitous in medical image classification problems. In such problems, it is often the case that rare classes associated to less prevalent diseases are severely under-represented in labeled databases, typically resulting in poor performance of machine learning algorithms due to overfitting in the learning process. In this paper, we propose a novel mechanism for sampling training data based on the popular MixUp regularization technique, which we refer to as Balanced-MixUp. In short, Balanced-MixUp simultaneously performs regular (i.e., instance-based) and balanced…
Xiaohe Healthcare, ByteDance（字节跳动小荷健康）；The University of Hong Kong（香港大学）
Medical image classification has been widely adopted in medical image analysis. However, due to the difficulty of collecting and labeling data in the medical area, medical image datasets are usually highly-imbalanced. To address this problem, previous works utilized class samples as prior for re-weighting or re-sampling but the feature representation is usually still not discriminative enough. In this paper, we adopt the contrastive learning to tackle the long-tailed medical imbalance problem. Specifically, we first propose the category prototype and adversarial proto-instance to generate repr…
Imbalanced training data is a significant challenge for medical image classification. In this study, we propose a novel Progressive Class-Center Triplet (PCCT) framework to alleviate the class imbalance issue particularly for diagnosis of rare diseases, mainly by carefully designing the triplet sampling strategy and the triplet loss formation. Specifically, the PCCT framework includes two successive stages. In the first stage, PCCT trains the diagnosis system via a class-balanced triplet loss to coarsely separate distributions of different classes. In the second stage, the PCCT framework furth…
Active Label Refinement for Robust Training of Imbalanced Medical Image Classification Tasks in the Presence of High Label Noise
Center for Imaging Science, Rochester Institute of Technology（罗切斯特理工学院成像科学中心）；University of Aberdeen（阿伯丁大学）；Biomedical Engineering, Rochester Institute of Technology（罗切斯特理工学院生物医学工程系）
Addressing Small and Imbalanced Medical Image Datasets Using Generative Models: A Comparative Study of DDPM and PGGANs with Random and Greedy K Sampling
Department of Computer Science, Bowling Green State University（计算机科学系，布卢明顿州立大学）
Medical Image Classification on Imbalanced Data Using ProGAN and SMA-Optimized ResNet: Application to COVID-19
Department of Mechatronics, School of Intelligent Systems College of Interdisciplinary Science and Technology University of Tehran（塔里克大学机械工程系、智能系统学院跨学科科学与技术学院）；School of Science and Technology Faculty of Science, Agriculture, Business and Law University of New England（新英格兰大学科学与技术学院科学、农业、商业与法学院）
Edith Cowan University（伊迪丝·考恩大学）；University of Western Australia（西澳大学）
Annotating medical images for disease detection is often tedious and expensive. Moreover, the available training samples for a given task are generally scarce and imbalanced. These conditions are not conducive for learning effective deep neural models. Hence, it is common to 'transfer' neural networks trained on natural images to the medical image domain. However, this paradigm lacks in performance due to the large domain gap between the natural and medical image data. To address that, we propose a novel concept of Pre-text Representation Transfer (PRT). In contrast to the conventional transfe…
Technical University of Munich (TUM)（慕尼黑工业大学）；Kyung Hee University（庆熙大学）；University of Munich (LMU)（慕尼黑大学）
One challenging property lurking in medical datasets is the imbalanced data distribution, where the frequency of the samples between the different classes is not balanced. Training a model on an imbalanced dataset can introduce unique challenges to the learning problem where a model is biased towards the highly frequent class. Many methods are proposed to tackle the distributional differences and the imbalanced problem. However, the impact of these approaches on the learned features is not well studied. In this paper, we look deeper into the internal units of neural networks to observe how han…
Class Attention to Regions of Lesion for Imbalanced Medical Image Recognition
Hong Kong University of Science and Technology（香港科技大学）；Sun Yat-sen University（中山大学）；Southern University of Science and Technology（南方科技大学）；Peng Cheng Laboratory（鹏城实验室）
Deep learning-based image processing is capable of creating highly appealing results. However, it is still widely considered as a "blackbox" transformation. In medical imaging, this lack of comprehensibility of the results is a sensitive issue. The integration of known operators into the deep learning environment has proven to be advantageous for the comprehensibility and reliability of the computations. Consequently, we propose the use of the locally linear guided filter in combination with a learned guidance map for general purpose medical image processing. The output images are only process…
Australian Institute for Machine Learning（澳大利亚机器学习研究所）；University of Adelaide（阿德莱德大学）；Otto von Guericke University Magdeburg（马格德堡奥托·冯·格里克大学）
Real-world large-scale medical image analysis (MIA) datasets have three challenges: 1) they contain noisy-labelled samples that affect training convergence and generalisation, 2) they usually have an imbalanced distribution of samples per class, and 3) they normally comprise a multi-label problem, where samples can have multiple diagnoses. Current approaches are commonly trained to solve a subset of those problems, but we are unaware of methods that address the three problems simultaneously. In this paper, we propose a new training module called Non-Volatile Unbiased Memory (NVUM), which non-v…
The Chinese University of Hong Kong（香港中文大学）；The University of Hong Kong（香港大学）
Deep neural networks have achieved remarkable success in a wide variety of natural image and medical image computing tasks. However, these achievements indispensably rely on accurately annotated training data. If encountering some noisy-labeled images, the network training procedure would suffer from difficulties, leading to a sub-optimal classifier. This problem is even more severe in the medical image analysis field, as the annotation quality of medical images heavily relies on the expertise and experience of annotators. In this paper, we propose a novel collaborative training paradigm with …
Distillation Learning Guided by Image Reconstruction for One-Shot Medical Image Segmentation
School of Artificial Intelligence, Beijing University of Posts and Telecommunications（北京邮电大学人工智能学院）；Peking University Third Hospital（北京大学第三医院）；Beijing Children’s Hospital, Capital Medical University（首都医科大学附属北京儿童医院）；Duke University（杜克大学）
School of Professional Studies（专业研究学院）；Northwestern University（西北大学）
Artificial intelligence (AI) is disrupting the medical field as advances in modern technology allow common household computers to learn anatomical and pathological features that distinguish between healthy and disease with the accuracy of highly specialized, trained physicians. Computer vision AI applications use medical imaging, such as lung chest X-Rays (LCXRs), to facilitate diagnoses by providing second-opinions in addition to a physician's or radiologist's interpretation. Considering the advent of the current Coronavirus disease (COVID-19) pandemic, LCXRs may provide rapid insights to ind…
Beyond the common difficulties faced in the natural image captioning, medical report generation specifically requires the model to describe a medical image with a fine-grained and semantic-coherence paragraph that should satisfy both medical commonsense and logic. Previous works generally extract the global image features and attempt to generate a paragraph that is similar to referenced reports; however, this approach has two limitations. Firstly, the regions of primary interest to radiologists are usually located in a small area of the global image, meaning that the remainder parts of the ima…
VGS-ATD: Robust Distributed Learning for Multi-Label Medical Image Classification Under Heterogeneous and Imbalanced Conditions
School of Computer Science, Queensland University of Technology（计算机科学学院，昆士兰理工大学）；Centre for Data Science, Queensland University of Technology（数据科学中心，昆士兰理工大学）；School of Mechanical, Medical, and Process Engineering, Queensland University of Technology（机械、医学和工艺工程学院，昆士兰理工大学）；Signal Processing, Artificial Intelligence and Vision Technologies (SAIVT), Queensland University of Technology（信号处理、人工智能和视觉技术（SAIVT），昆士兰理工大学）
Australian Institute for Machine Learning, University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；Universität Ulm（乌尔姆大学）
Effective semi-supervised learning (SSL) in medical image analysis (MIA) must address two challenges: 1) work effectively on both multi-class (e.g., lesion classification) and multi-label (e.g., multiple-disease diagnosis) problems, and 2) handle imbalanced learning (because of the high variance in disease prevalence). One strategy to explore in SSL MIA is based on the pseudo labelling strategy, but it has a few shortcomings. Pseudo-labelling has in general lower accuracy than consistency learning, it is not specifically designed for both multi-class and multi-label problems, and it can be cha…
Wayne State University（韦恩州立大学）；Michigan State University（密歇根州立大学）
This work tackles a central machine learning problem of performance degradation on out-of-distribution (OOD) test sets. The problem is particularly salient in medical imaging based diagnosis system that appears to be accurate but fails when tested in new hospitals/datasets. Recent studies indicate the system might learn shortcut and non-relevant features instead of generalizable features, so-called good features. We hypothesize that adversarial training can eliminate shortcut features whereas saliency guided training can filter out non-relevant features; both are nuisance features accounting f…
Text-guided Foundation Model Adaptation for Long-Tailed Medical Image Classification
Southern University of Science and Technology（南方科技大学）；The University of Hong Kong（香港大学）；University of British Columbia（不列颠哥伦比亚大学）；Jiaxing Research Institute, Southern University of Science and Technology（南方科技大学嘉兴研究院）
Sam-Guided Enhanced Fine-Grained Encoding with Mixed Semantic Learning for Medical Image Captioning
Zhejiang University-University of Illinois Urbana-Champaign Institute, Zhejiang University（浙江大学-伊利诺伊大学厄巴纳-香槟分校联合学院，浙江大学）；Sir Run Run Shaw Hospital, Zhejiang University School of Medicine（浙江大学医学院附属邵逸夫医院）
Medical images are widely used in clinical practice for diagnosis. Automatically generating interpretable medical reports can reduce radiologists' burden and facilitate timely care. However, most existing approaches to automatic report generation require sufficient labeled data for training. In addition, the learned model can only generate reports for the training classes, lacking the ability to adapt to previously unseen novel diseases. To this end, we propose a lesion guided explainable few weak-shot medical report generation framework that learns correlation between seen and novel classes t…
Generative Text-Guided 3D Vision-Language Pretraining for Unified Medical Image Segmentation
University of Science and Technology of China（中国科学技术大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）；Imperial College London（帝国理工学院）
LLM-Guided Diagnostic Evidence Alignment for Medical Vision-Language Pretraining under Limited Pairing
Institute of Intelligent Information Processing, Shanxi University（山西大学智能信息处理研究所）；Alliance Manchester Business School, The University of Manchester（曼彻斯特大学阿利安斯商学院）；Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（香港科技大学计算机科学与工程系）
Eye-gaze Guided Multi-modal Alignment for Medical Representation Learning
School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；City University of Hong Kong（香港城市大学）；The University of Texas at Arlington（德克萨斯大学阿灵顿分校）；School of Biomedical Engineering, ShanghaiTech University（上海科技大学生物医学工程学院）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能医疗科技有限公司）；Shanghai Clinical Research and Trial Center（上海临床研究与试验中心）；University of Georgia（佐治亚大学）；Massachusetts General Hospital（麻省总医院）
