Computation-Efficient Era: A Comprehensive Survey of State Space Models in Medical Image Analysis
School of Biomedical Engineering（生物医学工程学院）；University of British Columbia（不列颠哥伦比亚大学）；Tarbiat Modares University（塔比亚特·莫达雷斯大学）；University of Regensburg（雷根斯堡大学）；South Dakota State University（南达科他州立大学）；RWTH Aachen University（亚琛工业大学）；Lunit Inc.（Lunit公司）；Northwestern University（西北大学）；Fraunhofer Institute for Digital Medicine（弗劳恩霍夫数字医学研究院）
Moein Heidari, Sina Ghorbani Kolahi, Sanaz Karimijafarbigloo, Bobby Azad, Afshin Bozorgpour, Soheila Hatami, Reza Azad, Ali Diba, Ulas Bagci, Dorit Merhof, Ilker Hacihaliloglu
备注：
This is the first version of our survey, and the paper is currently under review
Sequence modeling plays a vital role across various domains, with recurrent neural networks being historically the predominant method of performing these tasks. However, the emergence of transformers has altered this paradigm due to their superior performance. Built upon these advances, transformers have conjoined CNNs as two leading foundational models for learning visual representations. However, transformers are hindered by the $\mathcal{O}(N^2)$ complexity of their attention mechanisms, while CNNs lack global receptive fields and dynamic weight allocation. State Space Models (SSMs), specifically the \textit{\textbf{Mamba}} model with selection mechanisms and hardware-aware architecture, have garnered immense interest lately in sequential modeling and visual representation learning, challenging the dominance of transformers by providing infinite context lengths and offering substantial efficiency maintaining linear complexity in the input sequence. Capitalizing on the advances in computer vision, medical imaging has heralded a new epoch with Mamba models. Intending to help researchers navigate the surge, this survey seeks to offer an encyclopedic review of Mamba models in medical imaging. Specifically, we start with a comprehensive theoretical review forming the basis of SSMs, including Mamba architecture and its alternatives for sequence modeling paradigms in this context. Next, we offer a structured classification of Mamba models in the medical field and introduce a diverse categorization scheme based on their application, imaging modalities, and targeted organs. Finally, we summarize key challenges, discuss different future research directions of the SSMs in the medical domain, and propose several directions to fulfill the demands of this field. In addition, we have compiled the studies discussed in this paper along with their open-source implementations on our GitHub repository.
In the realm of medical imaging, the training of machine learning models necessitates a large and varied training dataset to ensure robustness and interoperability. However, acquiring such diverse and heterogeneous data can be difficult due to the need for expert labeling of each image and privacy concerns associated with medical data. To circumvent these challenges, data augmentation has emerged as a promising and cost-effective technique for increasing the size and diversity of the training dataset. In this study, we provide a comprehensive review of the specific data augmentation techniques…
The Era of Foundation Models in Medical Imaging is Approaching : A Scoping Review of the Clinical Value of Large-Scale Generative AI Applications in Radiology
De-Identification of Medical Imaging Data: A Comprehensive Tool for Ensuring Patient Privacy
Institute for AI in Medicine (IKIM)（人工智能医学研究所（IKIM））；University Hospital Essen（埃森大学医院）；Cancer Research Center Cologne Essen (CCCE)（科隆-埃森癌症研究中心（CCCE））；University Medicine Essen（埃森大学医学部）；German Cancer Consortium (DKTK)（德国癌症联盟（DKTK））；Technical University Dortmund（多特蒙德工业大学）
Osaka Metropolitan University（大阪公立大学）；Smart Life Science Lab, Center for Health Science Innovation, Osaka Metropolitan University（大阪公立大学健康科学创新中心智能生命科学实验室）
The goal of our research is to create a comprehensive and flexible library that is easy to use for medical imaging research, and capable of handling grayscale images, multiple inputs (both images and tabular data), and multi-label tasks. We have named it Nervus. Based on the PyTorch library, which is suitable for AI for research purposes, we created a four-part model to handle comprehensive inputs and outputs. Nervus consists of four parts. First is the dataloader, then the feature extractor, the feature mixer, and finally the classifier. The dataloader preprocesses the input data, the feature…
Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）；Cincinnati Children’s Hospital Medical Center（辛辛那提儿童医院医疗中心）；University of Cincinnati College of Medicine（辛辛那提大学医学院）；University of Cincinnati（辛辛那提大学）
Image registration is a critical component in the applications of various medical image analyses. In recent years, there has been a tremendous surge in the development of deep learning (DL)-based medical image registration models. This paper provides a comprehensive review of medical image registration. Firstly, a discussion is provided for supervised registration categories, for example, fully supervised, dual supervised, and weakly supervised registration. Next, similarity-based as well as generative adversarial network (GAN)-based registration are presented as part of unsupervised registrat…
ETH Zurich（苏黎世联邦理工学院）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；University Hospital of Tübingen（蒂宾根大学医院）；Kantonsspital Winterthur（温特图尔州立医院）；GlaxoSmithKline（葛兰素史克）
Privacy concerns around sharing personally identifiable information are a major practical barrier to data sharing in medical research. However, in many cases, researchers have no interest in a particular individual's information but rather aim to derive insights at the level of cohorts. Here, we utilize Generative Adversarial Networks (GANs) to create derived medical imaging datasets consisting entirely of synthetic patient data. The synthetic images ideally have, in aggregate, similar statistical properties to those of a source dataset but do not contain sensitive personal information. We ass…
University of Electronic Science and Technology of China（电子科技大学）；SenseTime Research（商汤科技研究院）；University Hospitals Leuven（鲁汶大学医院）；King’s College London（伦敦国王学院）；University College London（伦敦大学学院）
Accurate medical image segmentation is essential for diagnosis and treatment planning of diseases. Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they are still challenged by complicated conditions where the segmentation target has large variations of position, shape and scale, and existing CNNs have a poor explainability that limits their application to clinical decisions. In this work, we make extensive use of multiple attentions in a CNN architecture and propose a comprehensive attention-based CNN (CA-Net) f…
Continual Learning in Medical Image Analysis: A Comprehensive Review of Recent Advancements and Future Prospects
Faculty of Informatics and Data Science, University of Regensburg（雷根斯堡大学信息学与数据科学学院）；University of California, Davis（加州大学戴维斯分校）；Motilal Nehru National Institute of Technology Allahabad（莫蒂拉尔·尼赫鲁国立阿拉哈巴德技术学院）；Faculty of Electrical Engineering and Information Technology, RWTH Aachen University（亚琛工业大学电气工程与信息技术学院）；Fraunhofer Institute for Digital Medicine MEVIS（弗劳恩霍夫数字医学MEVIS研究所）
Federated Learning for Medical Image Classification: A Comprehensive Benchmark
School of Electronic and Computer Engineering, Peking University（北京大学电子与计算机工程学院）；Shien-Ming Wu School of Intelligent Engineering, South China University of Technology（华南理工大学吴贤铭智能工程学院）
A comprehensive survey on deep active learning in medical image analysis
Fudan University（复旦大学）；School of Basic Medical Sciences, Fudan University（复旦大学基础医学院）；King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学）
Colorectal Polyp Segmentation in the Deep Learning Era: A Comprehensive Survey
Southwest Jiaotong University（西南交通大学）；China University of Petroleum (Qingdao)（中国石油大学（青岛））；Beihang University（北京航空航天大学）；Case Western Reserve University（凯斯西储大学）
MONICA: Benchmarking on Long-tailed Medical Image Classification
Monash University（莫纳什大学）；University College London（伦敦大学学院）；Moorfields Eye Hospital（摩尔菲尔兹眼科医院）；Airdoc Research（Airdoc研究院）；Imperial College London（伦敦帝国理工学院）
Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）；University of Cincinnati College of Medicine（辛辛那提大学医学院）；University of Cincinnati（辛辛那提大学）
The outbreak of novel coronavirus disease (COVID- 19) has claimed millions of lives and has affected all aspects of human life. This paper focuses on the application of deep learning (DL) models to medical imaging and drug discovery for managing COVID-19 disease. In this article, we detail various medical imaging-based studies such as X-rays and computed tomography (CT) images along with DL methods for classifying COVID-19 affected versus pneumonia. The applications of DL techniques to medical images are further described in terms of image localization, segmentation, registration, and classifi…
Modality-Projection Universal Model for Comprehensive Full-Body Medical Imaging Segmentation
the Institute of Medical Technology and National Biomedical Imaging Center, Peking University（北京大学医学技术研究所暨国家生物医学成像中心）；The First Affiliated Hospital of Shandong First Medical University & Shandong Provincial Qianfoshan Hospital（山东第一医科大学第一附属医院暨山东省千佛山医院）；Peking University Third Hospital（北京大学第三医院）；Guangdong Provincial People’s Hospital（广东省人民医院）；Peking University Cancer Hospital & Institute（北京大学肿瘤医院暨肿瘤研究所）
Comprehensive Review of Reinforcement Learning for Medical Ultrasound Imaging
Concordia University（康考迪亚大学）；Ecole de Technologie Superieure (ETS)（高等技术学院）；Laval University（拉瓦尔大学）；Khalifa University（哈利法大学）；Lebanese American University（黎巴嫩美国大学）
Deep Learning Approaches for Medical Imaging Under Varying Degrees of Label Availability: A Comprehensive Survey
University College Dublin（都柏林大学学院）；North China Institute of Aerospace Engineering（北华航天工业学院）；Hithink RoyalFlush Information Network Co., Ltd（同花顺信息网络股份有限公司）
Foundational Models in Medical Imaging: A Comprehensive Survey and Future Vision
South Dakota State University（南达科他州立大学）；RWTH Aachen University（亚琛工业大学）；University of Kentucky（肯塔基大学）；University of Regensburg（雷根斯堡大学）；Iran University of Science and Technology（伊朗科技大学）；Imperial College London（伦敦帝国理工学院）
A Comprehensive Survey of Mamba Architectures for Medical Image Analysis: Classification, Segmentation, Restoration and Beyond
Indian Institute of Technology Indore, India；R.M.D. Engineering College, Kavaraipettai, India；Jio Platforms Limited, India；Birla Institute of Technology \& Science Pilani, India
Advances in Medical Image Analysis with Vision Transformers: A Comprehensive Review
University of Regensburg（雷根斯堡大学）；Fraunhofer Institute for Digital Medicine MEVIS（弗劳恩霍夫数字医学研究所）；RWTH Aachen University（亚琛工业大学）；Iran University of Science and Technology（伊朗科技大学）；Shahid Beheshti University（沙希德·贝赫什提大学）
Federated Learning for Large Models in Medical Imaging: A Comprehensive Review
School of Cyber Science and Engineering, Sichuan University（四川大学网络科学与工程学院）；Sichuan Institute of Computer Sciences（四川省计算机科学研究院）；College of Computer Science, Sichuan University（四川大学计算机学院）；Department of Mathematics, The Chinese University of Hong Kong（香港中文大学数学系）；School of Electrical and Electronic Engineering, College of Engineering, Yonsei University（延世大学工程学院电子与电气工程系）
Prompt Mechanisms in Medical Imaging: A Comprehensive Survey
Faculty of Applied Sciences, Macao Polytechnic University（应用科学学院，澳门理工学院）；Netherlands Cancer Institute, Department of Radiology（荷兰癌症研究所，放射科）；Radboud University Medical Centre, Department of Radiology and Nuclear Medicine（拉德堡德大学医学中心，放射科和核医学科）；College of Aerospace Science and Engineering, National University of Defense Technology（航空航天科学与工程学院，国防科技大学）；Medical Department of Breast Cancer, Hunan Cancer Hospital（乳腺癌医学部，湖南癌症医院）；the Affiliated Cancer Hospital of Xiangya School of Medicine, Central South University（湘雅医学院附属肿瘤医院，中南大学）；Faculty of Biomedical Engineering, Eindhoven University of Technology（生物医学工程学院，埃因霍温理工大学）；Laboratory of Advanced Theranostic Materials and Technology, University of Chinese Academy of Sciences（先进诊疗材料与技术实验室，中国科学院大学）
National Heart and Lung Institute, Faculty of Medicine, Imperial College London（帝国理工学院医学院国家心肺研究所）；Department of Bioengineering, Faculty of Engineering, Imperial College London（帝国理工学院工程系生物工程系）；Royal Brompton Hospital（皇家布朗普顿医院）
Various structures in human physiology follow a treelike morphology, which often expresses complexity at very fine scales. Examples of such structures are intrathoracic airways, retinal blood vessels, and hepatic blood vessels. Large collections of 2D and 3D images have been made available by medical imaging modalities such as magnetic resonance imaging (MRI), computed tomography (CT), Optical coherence tomography (OCT) and ultrasound in which the spatial arrangement can be observed. Segmentation of these structures in medical imaging is of great importance since the analysis of the structure …
Comprehensive language-image pre-training for 3D medical image understanding
Microsoft（微软公司）；German Cancer Research Center (DKFZ)（德国癌症研究中心）；Department of Radiology, University of Cambridge and Cambridge University Hospitals NHS Foundation Trust（剑桥大学放射科及剑桥大学医院国家健康服务体系基金会信托）；Pattern Analysis and Learning Group, Heidelberg University Hospital（海德堡大学医院模式分析与学习组）；Department of Radiology, Mayo Clinic（梅奥诊所放射科）
The Impact of ChatGPT and LLMs on Medical Imaging Stakeholders: Perspectives and Use Cases
Swiss Federal Institute of Technology Lausanne (EPFL)（洛桑联邦理工学院）；Technical University of Munich（慕尼黑工业大学）；University of Zurich（苏黎世大学）；Boston College（波士顿学院）
Super-Generalist: Towards Comprehensive and Accurate Medical Image Understanding via Generalist-Specialist Synergy
DAMO Academy, Alibaba Group（达摩院，阿里巴巴集团）；Hupan Lab（湖畔实验室）；Department of Radiology, Ningbo No. 2 Hospital（宁波市第二医院放射科）；Department of Radiology, Guangdong Provincial People’s Hospital（广东省人民医院放射科）；Department of Radiology, Shanghai Institution of Pancreatic Disease（上海胰腺疾病研究所放射科）；Department of Radiology, Shengjing Hospital of China Medical University（中国医科大学附属盛京医院放射科）；The First Affiliated Hospital, Zhejiang University School of Medicine（浙江大学医学院附属第一医院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Benchmarking Real-World Medical Image Classification with Noisy Labels: Challenges, Practice, and Outlook
Japan Advanced Institute of Science and Technology（日本科学技术先进研究院）；University College London（伦敦大学学院）；The Hong Kong University of Science and Technology（香港科学大学）；University of Toyama（toyama大学）；Monash University（墨尔本大学）
Transformer, a model comprising attention-based encoder-decoder architecture, have gained prevalence in the field of natural language processing (NLP) and recently influenced the computer vision (CV) space. The similarities between computer vision and medical imaging, reviewed the question among researchers if the impact of transformers on computer vision be translated to medical imaging? In this paper, we attempt to provide a comprehensive and recent review on the application of transformers in medical imaging by; describing the transformer model comparing it with a diversity of convolutional…
How to build the best medical image segmentation algorithm using foundation models: a comprehensive empirical study with Segment Anything Model
Department of Electrical and Computer Engineering, Duke University, US（杜克大学电气与计算机工程系）；Department of Biostatistics & Bioinformatics, Duke University, US（杜克大学生物统计学与生物信息学系）；Department of Computer Science, Duke University, US（杜克大学计算机科学系）；Department of Radiology, Duke University, US（杜克大学放射学系）
Image registration is a fundamental medical image analysis task, and a wide variety of approaches have been proposed. However, only a few studies have comprehensively compared medical image registration approaches on a wide range of clinically relevant tasks. This limits the development of registration methods, the adoption of research advances into practice, and a fair benchmark across competing approaches. The Learn2Reg challenge addresses these limitations by providing a multi-task medical image registration data set for comprehensive characterisation of deformable registration algorithms. …
SemiSAM+: Rethinking Semi-Supervised Medical Image Segmentation in the Era of Foundation Models
Artificial Intelligence Innovation and Incubation Institute, Fudan University（复旦大学人工智能创新与孵化研究院）；Shanghai Academy of Artificial Intelligence for Science（上海人工智能科学研究院）；Huashan Hospital, Fudan University（复旦大学附属华山医院）；Human Phenome Institute, Fudan University（复旦大学人类表型组研究院）；School of Information Science and Engineering, Lanzhou University（兰州大学信息科学与工程学院）
School of Computer Sciences, Universiti Sains Malaysia（马来西亚理科大学计算机科学学院）；School of Network Communication, Zhejiang Yuexiu University（浙江越秀外国语学院网络传播学院）
Medical images often exhibit low and blurred contrast between lesions and surrounding tissues, with considerable variation in lesion edges and shapes even within the same disease, leading to significant challenges in segmentation. Therefore, precise segmentation of lesions has become an essential prerequisite for patient condition assessment and formulation of treatment plans. Significant achievements have been made in research related to the U-Net model in recent years. It improves segmentation performance and is extensively applied in the semantic segmentation of medical images to offer tech…
Diffusion Models for Medical Image Analysis: A Comprehensive Survey
Iran University of Science and Technology（伊朗科技大学）；Shahid Beheshti University（沙希德·贝赫什提大学）；RWTH Aachen University（亚琛工业大学）；Microsoft（微软）；University of British Columbia（不列颠哥伦比亚大学）；University of Regensburg（雷根斯堡大学）
Shiraz University of Technology（设拉子理工大学）；Shiraz University of Medical Science（设拉子医科大学）
Image-guided interventions are saving the lives of a large number of patients where the image registration problem should indeed be considered as the most complex and complicated issue to be tackled. On the other hand, the recently huge progress in the field of machine learning made by the possibility of implementing deep neural networks on the contemporary many-core GPUs opened up a promising window to challenge with many medical applications, where the registration is not an exception. In this paper, a comprehensive review on the state-of-the-art literature known as medical image registratio…
GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AI
Shanghai AI Laboratory（上海人工智能实验室）；University of Washington（华盛顿大学）；Monash University（莫纳什大学）；East China Normal University（华东师范大学）；Shanghai Jiao Tong University（上海交通大学）；Shenzhen Research Institute of Big Data（深圳大数据研究院）；University of Cambridge（剑桥大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Institute of Advanced Technology (SIAT), Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
COVID-19 (Coronavirus disease 2019) has been quickly spreading since its outbreak, impacting financial markets and healthcare systems globally. Countries all around the world have adopted a number of extraordinary steps to restrict the spreading virus, where early COVID-19 diagnosis is essential. Medical images such as X-ray images and Computed Tomography scans are becoming one of the main diagnostic tools to combat COVID-19 with the aid of deep learning-based systems. In this survey, we investigate the main contributions of deep learning applications using medical images in fighting against C…
From Model Based to Learned Regularization in Medical Image Registration: A Comprehensive Review
School of Computation, Information and Technology, Technical University of Munich (TUM)（慕尼黑工业大学计算、信息与技术学院）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich（亥姆霍兹慕尼黑生物医学成像机器学习研究所）；Department of Computing, Imperial College London（伦敦帝国理工学院计算机系）；School of Biomedical Engineering and Imaging Sciences, King’s College London（伦敦国王学院生物医学工程与影像科学学院）；School of Medicine, Klinikum rechts der Isar, Technical University of Munich（慕尼黑工业大学右岸医院医学院）
Shiraz University of Medical Sciences（设拉子医科大学）；University of Pittsburgh School of Medicine（匹兹堡大学医学院）；UPMC Hillman Cancer Center（UPMC希尔曼癌症中心）；Shiraz University of Technology（设拉子理工大学）；Mashhad University of Medical Science（马什哈德医科大学）
In medicine, image registration is vital in image-guided interventions and other clinical applications. However, it is a difficult subject to be addressed which by the advent of machine learning, there have been considerable progress in algorithmic performance has recently been achieved for medical image registration in this area. The implementation of deep neural networks provides an opportunity for some medical applications such as conducting image registration in less time with high accuracy, playing a key role in countering tumors during the operation. The current study presents a comprehe…
