The diffusion model has recently emerged as a potent approach in computer vision, demonstrating remarkable performances in the field of generative artificial intelligence. Capable of producing high-quality synthetic images, diffusion models have been successfully applied across a range of applications. However, a significant challenge remains with the high computational cost associated with training and generating these models. This study focuses on the efficiency and inference time of diffusion-based generative models, highlighting their applications in both natural and medical imaging. We present the most recent advances in diffusion models by categorizing them into three key models: the Denoising Diffusion Probabilistic Model (DDPM), the Latent Diffusion Model (LDM), and the Wavelet Diffusion Model (WDM). These models play a crucial role in medical imaging, where producing fast, reliable, and high-quality medical images is essential for accurate analysis of abnormalities and disease diagnosis. We first investigate the general framework of DDPM, LDM, and WDM and discuss the computational complexity gap filled by these models in natural and medical imaging. We then discuss the current limitations of these models as well as the opportunities and future research directions in medical imaging.
Diffusion Models for Medical Image Analysis: A Comprehensive Survey
Iran University of Science and Technology（伊朗科技大学）；Shahid Beheshti University（沙希德·贝赫什提大学）；RWTH Aachen University（亚琛工业大学）；Microsoft（微软）；University of British Columbia（不列颠哥伦比亚大学）；University of Regensburg（雷根斯堡大学）
In the realm of medical imaging, the training of machine learning models necessitates a large and varied training dataset to ensure robustness and interoperability. However, acquiring such diverse and heterogeneous data can be difficult due to the need for expert labeling of each image and privacy concerns associated with medical data. To circumvent these challenges, data augmentation has emerged as a promising and cost-effective technique for increasing the size and diversity of the training dataset. In this study, we provide a comprehensive review of the specific data augmentation techniques…
Osaka Metropolitan University（大阪公立大学）；Smart Life Science Lab, Center for Health Science Innovation, Osaka Metropolitan University（大阪公立大学健康科学创新中心智能生命科学实验室）
The goal of our research is to create a comprehensive and flexible library that is easy to use for medical imaging research, and capable of handling grayscale images, multiple inputs (both images and tabular data), and multi-label tasks. We have named it Nervus. Based on the PyTorch library, which is suitable for AI for research purposes, we created a four-part model to handle comprehensive inputs and outputs. Nervus consists of four parts. First is the dataloader, then the feature extractor, the feature mixer, and finally the classifier. The dataloader preprocesses the input data, the feature…
Modality-Projection Universal Model for Comprehensive Full-Body Medical Imaging Segmentation
the Institute of Medical Technology and National Biomedical Imaging Center, Peking University（北京大学医学技术研究所暨国家生物医学成像中心）；The First Affiliated Hospital of Shandong First Medical University & Shandong Provincial Qianfoshan Hospital（山东第一医科大学第一附属医院暨山东省千佛山医院）；Peking University Third Hospital（北京大学第三医院）；Guangdong Provincial People’s Hospital（广东省人民医院）；Peking University Cancer Hospital & Institute（北京大学肿瘤医院暨肿瘤研究所）
Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）；Cincinnati Children’s Hospital Medical Center（辛辛那提儿童医院医疗中心）；University of Cincinnati College of Medicine（辛辛那提大学医学院）；University of Cincinnati（辛辛那提大学）
Image registration is a critical component in the applications of various medical image analyses. In recent years, there has been a tremendous surge in the development of deep learning (DL)-based medical image registration models. This paper provides a comprehensive review of medical image registration. Firstly, a discussion is provided for supervised registration categories, for example, fully supervised, dual supervised, and weakly supervised registration. Next, similarity-based as well as generative adversarial network (GAN)-based registration are presented as part of unsupervised registrat…
School of Computer Sciences, Universiti Sains Malaysia（马来西亚理科大学计算机科学学院）；School of Network Communication, Zhejiang Yuexiu University（浙江越秀外国语学院网络传播学院）
Medical images often exhibit low and blurred contrast between lesions and surrounding tissues, with considerable variation in lesion edges and shapes even within the same disease, leading to significant challenges in segmentation. Therefore, precise segmentation of lesions has become an essential prerequisite for patient condition assessment and formulation of treatment plans. Significant achievements have been made in research related to the U-Net model in recent years. It improves segmentation performance and is extensively applied in the semantic segmentation of medical images to offer tech…
De-Identification of Medical Imaging Data: A Comprehensive Tool for Ensuring Patient Privacy
Institute for AI in Medicine (IKIM)（人工智能医学研究所（IKIM））；University Hospital Essen（埃森大学医院）；Cancer Research Center Cologne Essen (CCCE)（科隆-埃森癌症研究中心（CCCE））；University Medicine Essen（埃森大学医学部）；German Cancer Consortium (DKTK)（德国癌症联盟（DKTK））；Technical University Dortmund（多特蒙德工业大学）
Shiraz University of Technology（设拉子理工大学）；Shiraz University of Medical Science（设拉子医科大学）
Image-guided interventions are saving the lives of a large number of patients where the image registration problem should indeed be considered as the most complex and complicated issue to be tackled. On the other hand, the recently huge progress in the field of machine learning made by the possibility of implementing deep neural networks on the contemporary many-core GPUs opened up a promising window to challenge with many medical applications, where the registration is not an exception. In this paper, a comprehensive review on the state-of-the-art literature known as medical image registratio…
Image registration is a fundamental medical image analysis task, and a wide variety of approaches have been proposed. However, only a few studies have comprehensively compared medical image registration approaches on a wide range of clinically relevant tasks. This limits the development of registration methods, the adoption of research advances into practice, and a fair benchmark across competing approaches. The Learn2Reg challenge addresses these limitations by providing a multi-task medical image registration data set for comprehensive characterisation of deformable registration algorithms. …
Prompt Mechanisms in Medical Imaging: A Comprehensive Survey
Faculty of Applied Sciences, Macao Polytechnic University（应用科学学院，澳门理工学院）；Netherlands Cancer Institute, Department of Radiology（荷兰癌症研究所，放射科）；Radboud University Medical Centre, Department of Radiology and Nuclear Medicine（拉德堡德大学医学中心，放射科和核医学科）；College of Aerospace Science and Engineering, National University of Defense Technology（航空航天科学与工程学院，国防科技大学）；Medical Department of Breast Cancer, Hunan Cancer Hospital（乳腺癌医学部，湖南癌症医院）；the Affiliated Cancer Hospital of Xiangya School of Medicine, Central South University（湘雅医学院附属肿瘤医院，中南大学）；Faculty of Biomedical Engineering, Eindhoven University of Technology（生物医学工程学院，埃因霍温理工大学）；Laboratory of Advanced Theranostic Materials and Technology, University of Chinese Academy of Sciences（先进诊疗材料与技术实验室，中国科学院大学）
ETH Zurich（苏黎世联邦理工学院）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；University Hospital of Tübingen（蒂宾根大学医院）；Kantonsspital Winterthur（温特图尔州立医院）；GlaxoSmithKline（葛兰素史克）
Privacy concerns around sharing personally identifiable information are a major practical barrier to data sharing in medical research. However, in many cases, researchers have no interest in a particular individual's information but rather aim to derive insights at the level of cohorts. Here, we utilize Generative Adversarial Networks (GANs) to create derived medical imaging datasets consisting entirely of synthetic patient data. The synthetic images ideally have, in aggregate, similar statistical properties to those of a source dataset but do not contain sensitive personal information. We ass…
COVID-19 (Coronavirus disease 2019) has been quickly spreading since its outbreak, impacting financial markets and healthcare systems globally. Countries all around the world have adopted a number of extraordinary steps to restrict the spreading virus, where early COVID-19 diagnosis is essential. Medical images such as X-ray images and Computed Tomography scans are becoming one of the main diagnostic tools to combat COVID-19 with the aid of deep learning-based systems. In this survey, we investigate the main contributions of deep learning applications using medical images in fighting against C…
University of Electronic Science and Technology of China（电子科技大学）；SenseTime Research（商汤科技研究院）；University Hospitals Leuven（鲁汶大学医院）；King’s College London（伦敦国王学院）；University College London（伦敦大学学院）
Accurate medical image segmentation is essential for diagnosis and treatment planning of diseases. Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they are still challenged by complicated conditions where the segmentation target has large variations of position, shape and scale, and existing CNNs have a poor explainability that limits their application to clinical decisions. In this work, we make extensive use of multiple attentions in a CNN architecture and propose a comprehensive attention-based CNN (CA-Net) f…
Continual Learning in Medical Image Analysis: A Comprehensive Review of Recent Advancements and Future Prospects
Faculty of Informatics and Data Science, University of Regensburg（雷根斯堡大学信息学与数据科学学院）；University of California, Davis（加州大学戴维斯分校）；Motilal Nehru National Institute of Technology Allahabad（莫蒂拉尔·尼赫鲁国立阿拉哈巴德技术学院）；Faculty of Electrical Engineering and Information Technology, RWTH Aachen University（亚琛工业大学电气工程与信息技术学院）；Fraunhofer Institute for Digital Medicine MEVIS（弗劳恩霍夫数字医学MEVIS研究所）
Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）；University of Cincinnati College of Medicine（辛辛那提大学医学院）；University of Cincinnati（辛辛那提大学）
The outbreak of novel coronavirus disease (COVID- 19) has claimed millions of lives and has affected all aspects of human life. This paper focuses on the application of deep learning (DL) models to medical imaging and drug discovery for managing COVID-19 disease. In this article, we detail various medical imaging-based studies such as X-rays and computed tomography (CT) images along with DL methods for classifying COVID-19 affected versus pneumonia. The applications of DL techniques to medical images are further described in terms of image localization, segmentation, registration, and classifi…
GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AI
Shanghai AI Laboratory（上海人工智能实验室）；University of Washington（华盛顿大学）；Monash University（莫纳什大学）；East China Normal University（华东师范大学）；Shanghai Jiao Tong University（上海交通大学）；Shenzhen Research Institute of Big Data（深圳大数据研究院）；University of Cambridge（剑桥大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Institute of Advanced Technology (SIAT), Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Computation-Efficient Era: A Comprehensive Survey of State Space Models in Medical Image Analysis
School of Biomedical Engineering（生物医学工程学院）；University of British Columbia（不列颠哥伦比亚大学）；Tarbiat Modares University（塔比亚特·莫达雷斯大学）；University of Regensburg（雷根斯堡大学）；South Dakota State University（南达科他州立大学）；RWTH Aachen University（亚琛工业大学）；Lunit Inc.（Lunit公司）；Northwestern University（西北大学）；Fraunhofer Institute for Digital Medicine（弗劳恩霍夫数字医学研究院）
MONICA: Benchmarking on Long-tailed Medical Image Classification
Monash University（莫纳什大学）；University College London（伦敦大学学院）；Moorfields Eye Hospital（摩尔菲尔兹眼科医院）；Airdoc Research（Airdoc研究院）；Imperial College London（伦敦帝国理工学院）
From Model Based to Learned Regularization in Medical Image Registration: A Comprehensive Review
School of Computation, Information and Technology, Technical University of Munich (TUM)（慕尼黑工业大学计算、信息与技术学院）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich（亥姆霍兹慕尼黑生物医学成像机器学习研究所）；Department of Computing, Imperial College London（伦敦帝国理工学院计算机系）；School of Biomedical Engineering and Imaging Sciences, King’s College London（伦敦国王学院生物医学工程与影像科学学院）；School of Medicine, Klinikum rechts der Isar, Technical University of Munich（慕尼黑工业大学右岸医院医学院）
Shiraz University of Medical Sciences（设拉子医科大学）；University of Pittsburgh School of Medicine（匹兹堡大学医学院）；UPMC Hillman Cancer Center（UPMC希尔曼癌症中心）；Shiraz University of Technology（设拉子理工大学）；Mashhad University of Medical Science（马什哈德医科大学）
In medicine, image registration is vital in image-guided interventions and other clinical applications. However, it is a difficult subject to be addressed which by the advent of machine learning, there have been considerable progress in algorithmic performance has recently been achieved for medical image registration in this area. The implementation of deep neural networks provides an opportunity for some medical applications such as conducting image registration in less time with high accuracy, playing a key role in countering tumors during the operation. The current study presents a comprehe…
Comprehensive Review of Reinforcement Learning for Medical Ultrasound Imaging
Concordia University（康考迪亚大学）；Ecole de Technologie Superieure (ETS)（高等技术学院）；Laval University（拉瓦尔大学）；Khalifa University（哈利法大学）；Lebanese American University（黎巴嫩美国大学）
COMSATS University Islamabad（COMSATS伊斯兰堡大学）；Virtual University of Pakistan（巴基斯坦虚拟大学）；Cardiff University（卡迪夫大学）
The deep learning-based analysis of medical images suffers from data scarcity because of high annotation costs and privacy concerns. Researchers in this domain have used transfer learning to avoid overfitting when using complex architectures. However, the domain differences between pre-training and downstream data hamper the performance of the downstream task. Some recent studies have successfully used domain-adaptive pre-training (DAPT) to address this issue. In DAPT, models are initialized with the generic dataset pre-trained weights, and further pre-training is performed using a moderately …
University of Central Florida（中佛罗里达大学）；Mayo Clinic Cancer Center（妙佑医疗国际癌症中心）
Adversarial learning has been proven to be effective for capturing long-range and high-level label consistencies in semantic segmentation. Unique to medical imaging, capturing 3D semantics in an effective yet computationally efficient way remains an open problem. In this study, we address this computational burden by proposing a novel projective adversarial network, called PAN, which incorporates high-level 3D information through 2D projections. Furthermore, we introduce an attention module into our framework that helps for a selective integration of global information directly from our segmen…
A review of uncertainty quantification in medical image analysis: probabilistic and non-probabilistic methods
Saw Swee Hock School of Public Health, National University of Singapore（新加坡国立大学苏瑞福公共卫生学院）；University of Rouen Normandy（鲁昂诺曼底大学）；Institute of Data Science, National University of Singapore（新加坡国立大学数据科学研究所）
Institute of Computing Technology Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Capital Medical University（首都医科大学）
It is necessary for clinicians to comprehensively analyze patient information from different sources. Medical image fusion is a promising approach to providing overall information from medical images of different modalities. However, existing medical image fusion approaches ignore the semantics of images, making the fused image difficult to understand. In this work, we propose a new evaluation index to measure the semantic loss of fused image, and put forward a Fusion W-Net (FW-Net) for multimodal medical image fusion. The experimental results are promising: the fused image generated by our ap…
National Heart and Lung Institute, Faculty of Medicine, Imperial College London（帝国理工学院医学院国家心肺研究所）；Department of Bioengineering, Faculty of Engineering, Imperial College London（帝国理工学院工程系生物工程系）；Royal Brompton Hospital（皇家布朗普顿医院）
Various structures in human physiology follow a treelike morphology, which often expresses complexity at very fine scales. Examples of such structures are intrathoracic airways, retinal blood vessels, and hepatic blood vessels. Large collections of 2D and 3D images have been made available by medical imaging modalities such as magnetic resonance imaging (MRI), computed tomography (CT), Optical coherence tomography (OCT) and ultrasound in which the spatial arrangement can be observed. Segmentation of these structures in medical imaging is of great importance since the analysis of the structure …
A Comprehensive Review of Techniques, Algorithms, Advancements, Challenges, and Clinical Applications of Multi-modal Medical Image Fusion for Improved Diagnosis
Interdisciplinary Research Center for Finance and Digital Economy, King Fahd University of Petroleum and Minerals（金融与数字经济交叉研究中心，国王法赫德石油和矿物大学）；Department of Software Engineering, Faculty of Information Technology, Al-Ahliyya Amman University（软件工程系，信息科技学院，阿尔阿赫利亚大学）；Department of Information Systems and Operations Management, King Fahd University of Petroleum and Minerals（信息系统与运营管理系，国王法赫德石油和矿物大学）；ADAPT Research Centre, School of Computer Science, University of Galway（ADAPT研究中心，计算机科学学院，Galway大学）；Department of Mechanical and Nuclear Engineering, Khalifa University（机械与核工程系，哈利法大学）
Neural Radiance Fields in Medical Imaging: A Survey
University at Albany, State University of New York（纽约州立大学奥尔巴尼分校）；Purdue University（普渡大学）；University of North Texas（北得克萨斯大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Rutgers University（罗格斯大学）；SenseBrain Research（SenseBrain研究院）；National Heart Centre Singapore（新加坡国家心脏中心）；Duke-National University Of Singapore（杜克-新加坡国立大学）
Multi-modality images have been widely used and provide comprehensive information for medical image analysis. However, acquiring all modalities among all institutes is costly and often impossible in clinical settings. To leverage more comprehensive multi-modality information, we propose a privacy secured decentralized multi-modality adaptive learning architecture named ModalityBank. Our method could learn a set of effective domain-specific modulation parameters plugged into a common domain-agnostic network. We demonstrate by switching different sets of configurations, the generator could outpu…
Eindhoven University of Technology（埃因霍温理工大学）；Shandong University of Science and Technology（山东科技大学）；Philips Research（飞利浦研究院）
Medical instrument detection is essential for computer-assisted interventions since it would facilitate the surgeons to find the instrument efficiently with a better interpretation, which leads to a better outcome. This article reviews medical instrument detection methods in the ultrasound-guided intervention. First, we present a comprehensive review of instrument detection methodologies, which include traditional non-data-driven methods and data-driven methods. The non-data-driven methods were extensively studied prior to the era of machine learning, i.e. data-driven approaches. We discuss th…
Emory University（埃默里大学）；Winship Cancer Institute, Emory University（埃默里大学温希普癌症研究所）
This paper presents a review of deep learning (DL) based medical image registration methods. We summarized the latest developments and applications of DL-based registration methods in the medical field. These methods were classified into seven categories according to their methods, functions and popularity. A detailed review of each category was presented, highlighting important contributions and identifying specific challenges. A short assessment was presented following the detailed review of each category to summarize its achievements and future potentials. We provided a comprehensive compar…
Medical image slice interpolation is an active field of research. The methods for this task can be categorized into two broad groups: intensity-based and object-based interpolation methods. While intensity-based methods are generally easier to perform and less computationally expensive, object-based methods are capable of producing more accurate results and account for deformable changes in the objects within the slices. In this paper, performance of two well-known object-based interpolation methods is analyzed and compared. Here, a deformable registration-based method specifically designed fo…
Multimodal Marvels of Deep Learning in Medical Diagnosis: A Comprehensive Review of COVID-19 Detection
Institute for Intelligent Systems Research and Innovation (ISSRI), Deakin University（迪肯大学智能系统研究与创新研究所）；University of New South Wales（新南威尔士大学）；Bangladesh University of Business and Technology（孟加拉国商业与技术大学）；Green University（绿色大学）；Imam Mohammad Ibn Saud Islamic University（伊玛目穆罕默德·本·沙特伊斯兰大学）；Islamic University（伊斯兰大学）；The University of Queensland（昆士兰大学）；Artificial Intelligence and Cyber Futures Institute, Charles Stuart University（查尔斯·斯图尔特大学人工智能与网络未来研究所）
