The generalization performance of deep learning models for medical image analysis often decreases on images collected with different devices for data acquisition, device settings, or patient population. A better understanding of the generalization capacity on new images is crucial for clinicians' trustworthiness in deep learning. Although significant research efforts have been recently directed toward establishing generalization bounds and complexity measures, still, there is often a significant discrepancy between the predicted and actual generalization performance. As well, related large empirical studies have been primarily based on validation with general-purpose image datasets. This paper presents an empirical study that investigates the correlation between 25 complexity measures and the generalization abilities of supervised deep learning classifiers for breast ultrasound images. The results indicate that PAC-Bayes flatness-based and path norm-based measures produce the most consistent explanation for the combination of models and data. We also investigate the use of multi-task classification and segmentation approach for breast images, and report that such learning approach acts as an implicit regularizer and is conducive toward improved generalization.
Nanyang Technological University（南洋理工大学）；City University of Hong Kong（香港城市大学）；Karolinska Institute（卡罗林斯卡学院）；Karolinska University Hospital（卡罗林斯卡大学医院）
Recently, we have witnessed great progress in the field of medical imaging classification by adopting deep neural networks. However, the recent advanced models still require accessing sufficiently large and representative datasets for training, which is often unfeasible in clinically realistic environments. When trained on limited datasets, the deep neural network is lack of generalization capability, as the trained deep neural network on data within a certain distribution (e.g. the data captured by a certain device vendor or patient population) may not be able to generalize to the data with a…
This work provides a strong baseline for the problem of multi-source multi-target domain adaptation and generalization in medical imaging. Using a diverse collection of ten chest X-ray datasets, we empirically demonstrate the benefits of training medical imaging deep learning models on varied patient populations for generalization to out-of-sample domains.
With the advents of deep learning, improved image classification with complex discriminative models has been made possible. However, such deep models with increased complexity require a huge set of labeled samples to generalize the training. Such classification models can easily overfit when applied for medical images because of limited training data, which is a common problem in the field of medical image analysis. This paper proposes and investigates a reinforced classifier for improving the generalization under a few available training data. Partially following the idea of reinforcement lea…
The recent achievements of Deep Learning rely on the test data being similar in distribution to the training data. In an ideal case, Deep Learning models would achieve Out-of-Distribution (OoD) Generalization, i.e. reliably make predictions on out-of-distribution data. Yet in practice, models usually fail to generalize well when facing a shift in distribution. Several methods were thereby designed to improve the robustness of the features learned by a model through Regularization- or Domain-Prediction-based schemes. Segmenting medical images such as MRIs of the hippocampus is essential for the…
Deep learning models perform best when tested on target (test) data domains whose distribution is similar to the set of source (train) domains. However, model generalization can be hindered when there is significant difference in the underlying statistics between the target and source domains. In this work, we adapt a domain generalization method based on a model-agnostic meta-learning framework to biomedical imaging. The method learns a domain-agnostic feature representation to improve generalization of models to the unseen test distribution. The method can be used for any imaging task, as it…
Towards a Universal 3D Medical Multi-modality Generalization via Learning Personalized Invariant Representation
Shanghai Academy of Artificial Intelligence for Science（上海人工智能科学研究院）；Xi’an Jiaotong-Liverpool University（西安交通大学利物浦大学）；University of Liverpool（利物浦大学）；AI 3 Fudan University（复旦大学）；Duke Kunshan University（杜克-昆山大学）；Zhongshan Hospital（中山医院）
Shanghai Jiao Tong University（上海交通大学）；Fudan University（复旦大学）；Obstetrics and Gynecology Hospital, Institute of Reproduction and Development（妇产科医院生殖与发育研究院）；Zhejiang University City College（浙江大学城市学院）；Deepwise AI Lab（深睿人工智能实验室）；the Second Affiliated Hospital of Zhejiang University（浙江大学医学院附属第二医院）；Peking University Third Hospital（北京大学第三医院）
Since data scarcity and data heterogeneity are prevailing for medical images, well-trained Convolutional Neural Networks (CNNs) using previous normalization methods may perform poorly when deployed to a new site. However, a reliable model for real-world clinical applications should be able to generalize well both on in-distribution (IND) and out-of-distribution (OOD) data (e.g., the new site data). In this study, we present a novel normalization technique called window normalization (WIN) to improve the model generalization on heterogeneous medical images, which is a simple yet effective alter…
Thomas Jefferson University Hospital（托马斯·杰斐逊大学医院）；Sidney Kimmel Jefferson Medical College（西德尼·金梅尔杰斐逊医学院）
The discussions around Artificial Intelligence (AI) and medical imaging are centered around the success of deep learning algorithms. As new algorithms enter the market, it is important for practicing radiologists to understand the pitfalls of various AI algorithms. This entails having a basic understanding of how algorithms are developed, the kind of data they are trained on, and the settings in which they will be deployed. As with all new technologies, use of AI should be preceded by a fundamental understanding of the risks and benefits to those it is intended to help. This case-based review …
University of Electronic Science and Technology of China（电子科技大学）；Shanghai AI Lab（上海人工智能实验室）；ShanghaiTech University（上海科技大学）；Shanghai Jiao Tong University（上海交通大学）
Efficiently utilizing discriminative features is crucial for convolutional neural networks to achieve remarkable performance in medical image segmentation and is also important for model generalization across multiple domains, where letting model recognize domain-specific and domain-invariant information among multi-site datasets is a reasonable strategy for domain generalization. Unfortunately, most of the recent disentangle networks are not directly adaptable to unseen-domain datasets because of the limitations of offered data distribution. To tackle this deficiency, we propose Contrastive D…
Xiamen University（厦门大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Deepwise AI Laboratory（深睿医疗人工智能实验室）
Medical imaging datasets usually exhibit domain shift due to the variations of scanner vendors, imaging protocols, etc. This raises the concern about the generalization capacity of machine learning models. Domain generalization (DG), which aims to learn a model from multiple source domains such that it can be directly generalized to unseen test domains, seems particularly promising to medical imaging community. To address DG, recent model-agnostic meta-learning (MAML) has been introduced, which transfers the knowledge from previous training tasks to facilitate the learning of novel testing tas…
Isfahan University of Technology（伊斯法罕理工大学）；University of Michigan（密歇根大学）；McMaster University（麦克马斯特大学）
Noise is an important factor that degrades the quality of medical images. Impulse noise is a common noise, which is caused by malfunctioning of sensor elements or errors in the transmission of images. In medical images due to presence of white foreground and black background, many pixels have intensities similar to impulse noise and distinction between noisy and regular pixels is difficult. In software techniques, the accuracy of the noise removal is more important than the algorithm's complexity. But for hardware implementation having a low complexity algorithm with an acceptable accuracy is …
HCDG: A Hierarchical Consistency Framework for Domain Generalization on Medical Image Segmentation
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；University of Cambridge（剑桥大学）；The Hong Kong University of Science and Technology（香港科技大学）；The University of Hong Kong（香港大学）
Indian Institute of Technology, Madras（马德拉斯印度理工学院）
Deep Neural Networks can generalize despite being significantly overparametrized. Recent research has tried to examine this phenomenon from various view points and to provide bounds on the generalization error or measures predictive of the generalization gap based on these viewpoints, such as norm-based, PAC-Bayes based, and margin-based analysis. In this work, we provide an interpretation of generalization from the perspective of quality of internal representations of deep neural networks, based on neuroscientific theories of how the human visual system creates invariant and untangled object …
Deep learning-based computer-aided diagnosis is gradually deployed to review and analyze medical images. However, this paradigm is restricted in real-world clinical applications due to the poor robustness and generalization. The issue is more sinister with a lack of training data. In this paper, we address the challenge from the transfer learning point of view. Different from the common setting that transferring knowledge from the natural image domain to the medical image domain, we find the knowledge from the same domain further boosts the model robustness and generalization. Therefore, we pr…
Prompt-driven Latent Domain Generalization for Medical Image Classification
Monash University（莫纳什大学）；Airdoc-Monash Research, Monash University（Airdoc-莫纳什研究，莫纳什大学）；Inception Institute of AI（Inception人工智能研究院）；University of Queensland Diamantina Institute, Dermatology Research Centre, The University of Queensland（昆士兰大学Diamantina研究所，皮肤病学研究中心，昆士兰大学）；Victorian Melanoma Service, Alfred Health（维多利亚黑色素瘤服务中心，阿尔弗雷德健康中心）
Georgia Institute of Technology（佐治亚理工学院）；TReNDS Center（TReNDS中心）；Delft University of Technology（代尔夫特理工大学）
Self-supervised learning has enabled significant improvements on natural image benchmarks. However, there is less work in the medical imaging domain in this area. The optimal models have not yet been determined among the various options. Moreover, little work has evaluated the current applicability limits of novel self-supervised methods. In this paper, we evaluate a range of current contrastive self-supervised methods on out-of-distribution generalization in order to evaluate their applicability to medical imaging. We show that self-supervised models are not as robust as expected based on the…
FedSemiDG: Domain Generalized Federated Semi-supervised Medical Image Segmentation
Medical Artificial Intelligence Lab, Westlake University（西交大学医学人工智能实验室）；Department of Information and Communication Engineering, School of Engineering, Institute of Science Tokyo（东京科学大学工程学院信息与通信工程系）；Department of Biomedical Engineering, The Chinese University of Hong Kong（香港中文大学生物医学工程系）
Reconstructing medical images from partial measurements is an important inverse problem in Computed Tomography (CT) and Magnetic Resonance Imaging (MRI). Existing solutions based on machine learning typically train a model to directly map measurements to medical images, leveraging a training dataset of paired images and measurements. These measurements are typically synthesized from images using a fixed physical model of the measurement process, which hinders the generalization capability of models to unknown measurement processes. To address this issue, we propose a fully unsupervised techniq…
Domain generalization in medical image classification is an important problem for trustworthy machine learning to be deployed in healthcare. We find that existing approaches for domain generalization which utilize ground-truth abnormality segmentations to control feature attributions have poor out-of-distribution (OOD) performance relative to the standard baseline of empirical risk minimization (ERM). We investigate what regions of an image are important for medical image classification and show that parts of the background, that which is not contained in the abnormality segmentation, provides…
Google Research（谷歌研究院）；Georgia Institute of Technology（佐治亚理工学院）；Northwestern University（西北大学）
Recent progress in Medical Artificial Intelligence (AI) has delivered systems that can reach clinical expert level performance. However, such systems tend to demonstrate sub-optimal "out-of-distribution" performance when evaluated in clinical settings different from the training environment. A common mitigation strategy is to develop separate systems for each clinical setting using site-specific data [1]. However, this quickly becomes impractical as medical data is time-consuming to acquire and expensive to annotate [2]. Thus, the problem of "data-efficient generalization" presents an ongoing …
South China University of Technology（华南理工大学）；Pazhou Lab（琶洲实验室）；Tencent AI Lab（腾讯人工智能实验室）
Deep learning based medical image diagnosis has shown great potential in clinical medicine. However, it often suffers two major difficulties in real-world applications: 1) only limited labels are available for model training, due to expensive annotation costs over medical images; 2) labeled images may contain considerable label noise (e.g., mislabeling labels) due to diagnostic difficulties of diseases. To address these, we seek to exploit rich labeled data from relevant domains to help the learning in the target task via {Unsupervised Domain Adaptation} (UDA). Unlike most UDA methods that rel…
University of Hamburg（汉堡大学）；University Medical Center Hamburg-Eppendorf（汉堡-埃彭多夫大学医学中心）
Precisely locating and segmenting medical instruments in images of minimally invasive surgeries, medical instrument segmentation, is an essential first step for several tasks in medical image processing. However, image degradations, small instruments, and the generalization between different surgery types make medical instrument segmentation challenging. To cope with these challenges, we adapt the object proposal generation system AttentionMask and propose a dedicated post-processing to select promising proposals. The results on the ROBUST-MIS Challenge 2019 show that our adapted AttentionMask…
the University of Hong Kong（香港大学）；China Agricultural University（中国农业大学）；Deepwise Healthcare（深睿医疗）
During clinical practice, radiologists often use attributes, e.g. morphological and appearance characteristics of a lesion, to aid disease diagnosis. Effectively modeling attributes as well as all relationships involving attributes could boost the generalization ability and verifiability of medical image diagnosis algorithms. In this paper, we introduce a hybrid neuro-probabilistic reasoning algorithm for verifiable attribute-based medical image diagnosis. There are two parallel branches in our hybrid algorithm, a Bayesian network branch performing probabilistic causal relationship reasoning a…
Iran University of Science and Technology（伊朗科技大学）；West Virginia University（西弗吉尼亚大学）
Convolutional Neural Networks (CNNs) have advanced existing medical systems for automatic disease diagnosis. However, there are still concerns about the reliability of deep medical diagnosis systems against the potential threats of adversarial attacks since inaccurate diagnosis could lead to disastrous consequences in the safety realm. In this study, we propose a highly robust yet efficient CNN-Transformer hybrid model which is equipped with the locality of CNNs as well as the global connectivity of vision Transformers. To mitigate the high quadratic complexity of the self-attention mechanism …
