Medical Image Analysis (MedIA) has emerged as a crucial tool in computer-aided diagnosis systems, particularly with the advancement of deep learning (DL) in recent years. However, well-trained deep models often experience significant performance degradation when deployed in different medical sites, modalities, and sequences, known as a domain shift issue. In light of this, Domain Generalization (DG) for MedIA aims to address the domain shift challenge by generalizing effectively and performing robustly across unknown data distributions. This paper presents the a comprehensive review of substantial developments in this area. First, we provide a formal definition of domain shift and domain generalization in medical field, and discuss several related settings. Subsequently, we summarize the recent methods from three viewpoints: data manipulation level, feature representation level, and model training level, and present some algorithms in detail for each viewpoints. Furthermore, we introduce the commonly used datasets. Finally, we summarize existing literature and present some potential research topics for the future. For this survey, we also created a GitHub project by collecting the supporting resources, at the link: https://github.com/Ziwei-Niu/DG_for_MedIA
Eindhoven University of Technology（埃因霍温理工大学）；Erasmus Medical Center（伊拉斯谟医学中心）；University of Copenhagen（哥本哈根大学）；University Medical Center Utrecht（乌得勒支大学医学中心）
Machine learning (ML) algorithms have made a tremendous impact in the field of medical imaging. While medical imaging datasets have been growing in size, a challenge for supervised ML algorithms that is frequently mentioned is the lack of annotated data. As a result, various methods which can learn with less/other types of supervision, have been proposed. We review semi-supervised, multiple instance, and transfer learning in medical imaging, both in diagnosis/detection or segmentation tasks. We also discuss connections between these learning scenarios, and opportunities for future research.
Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks and provide concise overviews of studies per application area. Open challenges and directions for future research are discussed.
CentraleSupelec（中央理工-高等电力学院）；INRIA（法国国家信息与自动化研究所）；Universite Paris-Saclay（巴黎萨克雷大学）；Imperial College London（帝国理工学院）；TheraPanacea
During the last decades, the research community of medical imaging has witnessed continuous advances in image registration methods, which pushed the limits of the state-of-the-art and enabled the development of novel medical procedures. A particular type of image registration problem, known as slice-to-volume registration, played a fundamental role in areas like image guided surgeries and volumetric image reconstruction. However, to date, and despite the extensive literature available on this topic, no survey has been written to discuss this challenging problem. This paper introduces the first…
Statistical machine learning algorithms have achieved state-of-the-art results on benchmark datasets, outperforming humans in many tasks. However, the out-of-distribution data and confounder, which have an unpredictable causal relationship, significantly degrade the performance of the existing models. Causal Representation Learning (CRL) has recently been a promising direction to address the causal relationship problem in vision understanding. This survey presents recent advances in CRL in vision. Firstly, we introduce the basic concept of causal inference. Secondly, we analyze the CRL theoret…
A comprehensive survey on deep active learning in medical image analysis
Fudan University（复旦大学）；School of Basic Medical Sciences, Fudan University（复旦大学基础医学院）；King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学）
Recent Advances in Medical Imaging Segmentation: A Survey
Junia, UMR 8520, CNRS, Centrale Lille, Univerity of Polytechnique Hauts-de-France（Junia研究院、法国国家科学研究中心、Centrale Lille、法国高等技术大学 Hauts-de-France）；Sorbonne Center for Artificial Intelligence (SCAI), Abu Dhabi, UAE（索邦人工智能中心（SCAI））
School of Biomedical Engineering, Southern Medical University（南方医科大学生物医学工程学院）；School of Biomedical Engineering, Shanghai Jiao Tong University（上海交通大学生物医学工程学院）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能医疗科技有限公司）；Shanghai Clinical Research and Trial Center（上海临床研究与试验中心）
Shanghai Jiao Tong University（上海交通大学）；Fudan University（复旦大学）；Obstetrics and Gynecology Hospital, Institute of Reproduction and Development（妇产科医院生殖与发育研究院）；Zhejiang University City College（浙江大学城市学院）；Deepwise AI Lab（深睿人工智能实验室）；the Second Affiliated Hospital of Zhejiang University（浙江大学医学院附属第二医院）；Peking University Third Hospital（北京大学第三医院）
Since data scarcity and data heterogeneity are prevailing for medical images, well-trained Convolutional Neural Networks (CNNs) using previous normalization methods may perform poorly when deployed to a new site. However, a reliable model for real-world clinical applications should be able to generalize well both on in-distribution (IND) and out-of-distribution (OOD) data (e.g., the new site data). In this study, we present a novel normalization technique called window normalization (WIN) to improve the model generalization on heterogeneous medical images, which is a simple yet effective alter…
This work provides a strong baseline for the problem of multi-source multi-target domain adaptation and generalization in medical imaging. Using a diverse collection of ten chest X-ray datasets, we empirically demonstrate the benefits of training medical imaging deep learning models on varied patient populations for generalization to out-of-sample domains.
University of Electronic Science and Technology of China（电子科技大学）；Shanghai AI Lab（上海人工智能实验室）；ShanghaiTech University（上海科技大学）；Shanghai Jiao Tong University（上海交通大学）
Efficiently utilizing discriminative features is crucial for convolutional neural networks to achieve remarkable performance in medical image segmentation and is also important for model generalization across multiple domains, where letting model recognize domain-specific and domain-invariant information among multi-site datasets is a reasonable strategy for domain generalization. Unfortunately, most of the recent disentangle networks are not directly adaptable to unseen-domain datasets because of the limitations of offered data distribution. To tackle this deficiency, we propose Contrastive D…
Self-eXplainable AI for Medical Image Analysis: A Survey and New Outlooks
Hong Kong University of Science and Technology（香港科技大学）；Shenzhen MSU-BIT University（深圳北理莫斯科大学）；Harvard University（哈佛大学）；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute（香港科技大学深港协同创新研究院）
HCDG: A Hierarchical Consistency Framework for Domain Generalization on Medical Image Segmentation
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；University of Cambridge（剑桥大学）；The Hong Kong University of Science and Technology（香港科技大学）；The University of Hong Kong（香港大学）
Beihang University（北京航空航天大学）；Beijing Advanced Innovation Center for Big Data and Brain Computing（北京大数据与脑计算高精尖创新中心）；Hangzhou Innovation Institute of Beihang University（北京航空航天大学杭州创新研究院）；The University of Texas at Dallas（德克萨斯大学达拉斯分校）；University of Technology Sydney（悉尼科技大学）
Although deep learning models like CNNs have achieved great success in medical image analysis, the small size of medical datasets remains a major bottleneck in this area. To address this problem, researchers have started looking for external information beyond current available medical datasets. Traditional approaches generally leverage the information from natural images via transfer learning. More recent works utilize the domain knowledge from medical doctors, to create networks that resemble how medical doctors are trained, mimic their diagnostic patterns, or focus on the features or areas …
Foundational Models in Medical Imaging: A Comprehensive Survey and Future Vision
South Dakota State University（南达科他州立大学）；RWTH Aachen University（亚琛工业大学）；University of Kentucky（肯塔基大学）；University of Regensburg（雷根斯堡大学）；Iran University of Science and Technology（伊朗科技大学）；Imperial College London（伦敦帝国理工学院）
The texture is defined as spatial structure of the intensities of the pixels in an image that is repeated periodically in the whole image or regions, and makes the concept of the image. Texture, color and shape are three main components which are used by human visual system to recognize image contents. In this paper, first of all, efficient and updated texture analysis operators are survived with details. Next, some state-of-the-art methods are survived that use texture analysis in medical applications and disease diagnosis. Finally, different approaches are compared in terms of accuracy, data…
Medical Image Registration and Its Application in Retinal Images: A Review
Southern University of Science and Technology（南方科技大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Singapore Eye Research Institute（新加坡眼科研究所）
Nanyang Technological University（南洋理工大学）；City University of Hong Kong（香港城市大学）；Karolinska Institute（卡罗林斯卡学院）；Karolinska University Hospital（卡罗林斯卡大学医院）
Recently, we have witnessed great progress in the field of medical imaging classification by adopting deep neural networks. However, the recent advanced models still require accessing sufficiently large and representative datasets for training, which is often unfeasible in clinically realistic environments. When trained on limited datasets, the deep neural network is lack of generalization capability, as the trained deep neural network on data within a certain distribution (e.g. the data captured by a certain device vendor or patient population) may not be able to generalize to the data with a…
Medical image fusion is the process of registering and combining multiple images from single or multiple imaging modalities to improve the imaging quality and reduce randomness and redundancy in order to increase the clinical applicability of medical images for diagnosis and assessment of medical problems. Multi-modal medical image fusion algorithms and devices have shown notable achievements in improving clinical accuracy of decisions based on medical images. This review article provides a factual listing of methods and summarizes the broad scientific challenges faced in the field of medical …
Georgia Institute of Technology（佐治亚理工学院）；TReNDS Center（TReNDS中心）；Delft University of Technology（代尔夫特理工大学）
Self-supervised learning has enabled significant improvements on natural image benchmarks. However, there is less work in the medical imaging domain in this area. The optimal models have not yet been determined among the various options. Moreover, little work has evaluated the current applicability limits of novel self-supervised methods. In this paper, we evaluate a range of current contrastive self-supervised methods on out-of-distribution generalization in order to evaluate their applicability to medical imaging. We show that self-supervised models are not as robust as expected based on the…
Towards a Universal 3D Medical Multi-modality Generalization via Learning Personalized Invariant Representation
Shanghai Academy of Artificial Intelligence for Science（上海人工智能科学研究院）；Xi’an Jiaotong-Liverpool University（西安交通大学利物浦大学）；University of Liverpool（利物浦大学）；AI 3 Fudan University（复旦大学）；Duke Kunshan University（杜克-昆山大学）；Zhongshan Hospital（中山医院）
Every year physicians face an increasing demand of image-based diagnosis from patients, a problem that can be addressed with recent artificial intelligence methods. In this context, we survey works in the area of automatic report generation from medical images, with emphasis on methods using deep neural networks, with respect to: (1) Datasets, (2) Architecture Design, (3) Explainability and (4) Evaluation Metrics. Our survey identifies interesting developments, but also remaining challenges. Among them, the current evaluation of generated reports is especially weak, since it mostly relies on t…
MBZ University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Inception Institute of Artificial Intelligence（起源人工智能研究院）；Monash University（蒙纳士大学）；Australian National University（澳大利亚国立大学）；Linköping University（林雪平大学）；Agency for Science, Technology and Research（新加坡科技研究局）
Following unprecedented success on the natural language tasks, Transformers have been successfully applied to several computer vision problems, achieving state-of-the-art results and prompting researchers to reconsider the supremacy of convolutional neural networks (CNNs) as {de facto} operators. Capitalizing on these advances in computer vision, the medical imaging field has also witnessed growing interest for Transformers that can capture global context compared to CNNs with local receptive fields. Inspired from this transition, in this survey, we attempt to provide a comprehensive review of…
Implicit Neural Representation in Medical Imaging: A Comparative Survey
Iran University of Science and Technology（伊朗科技大学）；South Dakota State University（南达科他州立大学）；RWTH Aachen University（亚琛工业大学）；University of Regensburg（雷根斯堡大学）；Fraunhofer Institute for Digital Medicine MEVIS（弗劳恩霍夫数字医学MEVIS研究所）
FedSemiDG: Domain Generalized Federated Semi-supervised Medical Image Segmentation
Medical Artificial Intelligence Lab, Westlake University（西交大学医学人工智能实验室）；Department of Information and Communication Engineering, School of Engineering, Institute of Science Tokyo（东京科学大学工程学院信息与通信工程系）；Department of Biomedical Engineering, The Chinese University of Hong Kong（香港中文大学生物医学工程系）
Survey on Adversarial Attack and Defense for Medical Image Analysis: Methods and Challenges
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Guangdong Province Key Laboratory of Information Security Technology（广东省信息安全技术重点实验室）；Division of Life Science, Hong Kong University of Science and Technology（香港科技大学生命科学部）
Thomas Jefferson University Hospital（托马斯·杰斐逊大学医院）；Sidney Kimmel Jefferson Medical College（西德尼·金梅尔杰斐逊医学院）
The discussions around Artificial Intelligence (AI) and medical imaging are centered around the success of deep learning algorithms. As new algorithms enter the market, it is important for practicing radiologists to understand the pitfalls of various AI algorithms. This entails having a basic understanding of how algorithms are developed, the kind of data they are trained on, and the settings in which they will be deployed. As with all new technologies, use of AI should be preceded by a fundamental understanding of the risks and benefits to those it is intended to help. This case-based review …
Jordan University of Science and Technology（约旦科技大学）
The scarcity of high-quality annotated medical imaging datasets is a major problem that collides with machine learning applications in the field of medical imaging analysis and impedes its advancement. Self-supervised learning is a recent training paradigm that enables learning robust representations without the need for human annotation which can be considered an effective solution for the scarcity of annotated medical data. This article reviews the state-of-the-art research directions in self-supervised learning approaches for image data with a concentration on their applications in the fiel…
Xiamen University（厦门大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Deepwise AI Laboratory（深睿医疗人工智能实验室）
Medical imaging datasets usually exhibit domain shift due to the variations of scanner vendors, imaging protocols, etc. This raises the concern about the generalization capacity of machine learning models. Domain generalization (DG), which aims to learn a model from multiple source domains such that it can be directly generalized to unseen test domains, seems particularly promising to medical imaging community. To address DG, recent model-agnostic meta-learning (MAML) has been introduced, which transfers the knowledge from previous training tasks to facilitate the learning of novel testing tas…
With the advents of deep learning, improved image classification with complex discriminative models has been made possible. However, such deep models with increased complexity require a huge set of labeled samples to generalize the training. Such classification models can easily overfit when applied for medical images because of limited training data, which is a common problem in the field of medical image analysis. This paper proposes and investigates a reinforced classifier for improving the generalization under a few available training data. Partially following the idea of reinforcement lea…
Deep Learning Approaches for Medical Imaging Under Varying Degrees of Label Availability: A Comprehensive Survey
University College Dublin（都柏林大学学院）；North China Institute of Aerospace Engineering（北华航天工业学院）；Hithink RoyalFlush Information Network Co., Ltd（同花顺信息网络股份有限公司）
University of Copenhagen（哥本哈根大学）；McGill Centre for Integrative Neuroscience（麦吉尔整合神经科学中心）；Erasmus Medical Center（伊拉斯谟医学中心）；Technische Universität Ilmenau（伊尔默瑙工业大学）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）；Delft University of Technology（代尔夫特理工大学）；University of Nottingham（诺丁汉大学）；University of Oxford（牛津大学）；Zooniverse, University of Oxford（牛津大学Zooniverse）；Eindhoven University of Technology（埃因霍温理工大学）
Rapid advances in image processing capabilities have been seen across many domains, fostered by the application of machine learning algorithms to "big-data". However, within the realm of medical image analysis, advances have been curtailed, in part, due to the limited availability of large-scale, well-annotated datasets. One of the main reasons for this is the high cost often associated with producing large amounts of high-quality meta-data. Recently, there has been growing interest in the application of crowdsourcing for this purpose; a technique that has proven effective for creating large-s…
Image Sciences Institute, University Medical Center Utrecht（乌得勒支大学医学中心影像科学研究所）；Utrecht University（乌得勒支大学）
With an increase in deep learning-based methods, the call for explainability of such methods grows, especially in high-stakes decision making areas such as medical image analysis. This survey presents an overview of eXplainable Artificial Intelligence (XAI) used in deep learning-based medical image analysis. A framework of XAI criteria is introduced to classify deep learning-based medical image analysis methods. Papers on XAI techniques in medical image analysis are then surveyed and categorized according to the framework and according to anatomical location. The paper concludes with an outloo…
J. J. Strossmayer University of Osijek（奥西耶克J.J.斯特罗斯马耶大学）；Ghent University（根特大学）；imec（比利时微电子研究中心）
Atrial fibrillation (AF) is a cardiovascular disease identified as one of the main risk factors for stroke. The majority of strokes due to AF are caused by clots originating in the left atrial appendage (LAA). LAA occlusion is an effective procedure for reducing stroke risk. Planning the procedure using pre-procedural imaging and analysis has shown benefits. The analysis is commonly done by manually segmenting the appendage on 2D slices. Automatic LAA segmentation methods could save an expert's time and provide insightful 3D visualizations and accurate automatic measurements to aid in medical …
The recent achievements of Deep Learning rely on the test data being similar in distribution to the training data. In an ideal case, Deep Learning models would achieve Out-of-Distribution (OoD) Generalization, i.e. reliably make predictions on out-of-distribution data. Yet in practice, models usually fail to generalize well when facing a shift in distribution. Several methods were thereby designed to improve the robustness of the features learned by a model through Regularization- or Domain-Prediction-based schemes. Segmenting medical images such as MRIs of the hippocampus is essential for the…
Deep learning-based computer-aided diagnosis is gradually deployed to review and analyze medical images. However, this paradigm is restricted in real-world clinical applications due to the poor robustness and generalization. The issue is more sinister with a lack of training data. In this paper, we address the challenge from the transfer learning point of view. Different from the common setting that transferring knowledge from the natural image domain to the medical image domain, we find the knowledge from the same domain further boosts the model robustness and generalization. Therefore, we pr…
