A Survey of Crowdsourcing in Medical Image Analysis
University of Copenhagen（哥本哈根大学）；McGill Centre for Integrative Neuroscience（麦吉尔整合神经科学中心）；Erasmus Medical Center（伊拉斯谟医学中心）；Technische Universität Ilmenau（伊尔默瑙工业大学）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）；Delft University of Technology（代尔夫特理工大学）；University of Nottingham（诺丁汉大学）；University of Oxford（牛津大学）；Zooniverse, University of Oxford（牛津大学Zooniverse）；Eindhoven University of Technology（埃因霍温理工大学）
Silas Ørting, Andrew Doyle, Arno van Hilten, Matthias Hirth, Oana Inel, Christopher R. Madan, Panagiotis Mavridis, Helen Spiers, Veronika Cheplygina
Rapid advances in image processing capabilities have been seen across many domains, fostered by the application of machine learning algorithms to "big-data". However, within the realm of medical image analysis, advances have been curtailed, in part, due to the limited availability of large-scale, well-annotated datasets. One of the main reasons for this is the high cost often associated with producing large amounts of high-quality meta-data. Recently, there has been growing interest in the application of crowdsourcing for this purpose; a technique that has proven effective for creating large-scale datasets across a range of disciplines, from computer vision to astrophysics. Despite the growing popularity of this approach, there has not yet been a comprehensive literature review to provide guidance to researchers considering using crowdsourcing methodologies in their own medical imaging analysis. In this survey, we review studies applying crowdsourcing to the analysis of medical images, published prior to July 2018. We identify common approaches, challenges and considerations, providing guidance of utility to researchers adopting this approach. Finally, we discuss future opportunities for development within this emerging domain.
Imperial College London（伦敦帝国学院）；Aichi Cancer Center（爱知县癌症中心）；Nagoya University（名古屋大学）
To efficiently establish training databases for machine learning methods, collaborative and crowdsourcing platforms have been investigated to collectively tackle the annotation effort. However, when this concept is ported to the medical imaging domain, reading expertise will have a direct impact on the annotation accuracy. In this study, we examine the impact of expertise and the amount of available annotations on the accuracy outcome of a liver segmentation problem in an abdominal computed tomography (CT) image database. In controlled experiments, we study this impact for different types of w…
Delft University of Technology（代尔夫特理工大学）；Erasmus Medical Center - Sophia Children’s Hospital（伊拉斯谟医学中心-索菲亚儿童医院）；Erasmus Medical Center（伊拉斯谟医学中心）；University of Copenhagen（哥本哈根大学）
Measuring airways in chest computed tomography (CT) images is important for characterizing diseases such as cystic fibrosis, yet very time-consuming to perform manually. Machine learning algorithms offer an alternative, but need large sets of annotated data to perform well. We investigate whether crowdsourcing can be used to gather airway annotations which can serve directly for measuring the airways, or as training data for the algorithms. We generate image slices at known locations of airways and request untrained crowd workers to outline the airway lumen and airway wall. Our results show th…
Eindhoven University of Technology（埃因霍温理工大学）；Erasmus Medical Center（伊拉斯谟医学中心）；University of Copenhagen（哥本哈根大学）；University Medical Center Utrecht（乌得勒支大学医学中心）
Machine learning (ML) algorithms have made a tremendous impact in the field of medical imaging. While medical imaging datasets have been growing in size, a challenge for supervised ML algorithms that is frequently mentioned is the lack of annotated data. As a result, various methods which can learn with less/other types of supervision, have been proposed. We review semi-supervised, multiple instance, and transfer learning in medical imaging, both in diagnosis/detection or segmentation tasks. We also discuss connections between these learning scenarios, and opportunities for future research.
Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks and provide concise overviews of studies per application area. Open challenges and directions for future research are discussed.
CentraleSupelec（中央理工-高等电力学院）；INRIA（法国国家信息与自动化研究所）；Universite Paris-Saclay（巴黎萨克雷大学）；Imperial College London（帝国理工学院）；TheraPanacea
During the last decades, the research community of medical imaging has witnessed continuous advances in image registration methods, which pushed the limits of the state-of-the-art and enabled the development of novel medical procedures. A particular type of image registration problem, known as slice-to-volume registration, played a fundamental role in areas like image guided surgeries and volumetric image reconstruction. However, to date, and despite the extensive literature available on this topic, no survey has been written to discuss this challenging problem. This paper introduces the first…
Statistical machine learning algorithms have achieved state-of-the-art results on benchmark datasets, outperforming humans in many tasks. However, the out-of-distribution data and confounder, which have an unpredictable causal relationship, significantly degrade the performance of the existing models. Causal Representation Learning (CRL) has recently been a promising direction to address the causal relationship problem in vision understanding. This survey presents recent advances in CRL in vision. Firstly, we introduce the basic concept of causal inference. Secondly, we analyze the CRL theoret…
A comprehensive survey on deep active learning in medical image analysis
Fudan University（复旦大学）；School of Basic Medical Sciences, Fudan University（复旦大学基础医学院）；King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学）
Beihang University（北京航空航天大学）；Beijing Advanced Innovation Center for Big Data and Brain Computing（北京大数据与脑计算高精尖创新中心）；Hangzhou Innovation Institute of Beihang University（北京航空航天大学杭州创新研究院）；The University of Texas at Dallas（德克萨斯大学达拉斯分校）；University of Technology Sydney（悉尼科技大学）
Although deep learning models like CNNs have achieved great success in medical image analysis, the small size of medical datasets remains a major bottleneck in this area. To address this problem, researchers have started looking for external information beyond current available medical datasets. Traditional approaches generally leverage the information from natural images via transfer learning. More recent works utilize the domain knowledge from medical doctors, to create networks that resemble how medical doctors are trained, mimic their diagnostic patterns, or focus on the features or areas …
Recent Advances in Medical Imaging Segmentation: A Survey
Junia, UMR 8520, CNRS, Centrale Lille, Univerity of Polytechnique Hauts-de-France（Junia研究院、法国国家科学研究中心、Centrale Lille、法国高等技术大学 Hauts-de-France）；Sorbonne Center for Artificial Intelligence (SCAI), Abu Dhabi, UAE（索邦人工智能中心（SCAI））
School of Biomedical Engineering, Southern Medical University（南方医科大学生物医学工程学院）；School of Biomedical Engineering, Shanghai Jiao Tong University（上海交通大学生物医学工程学院）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能医疗科技有限公司）；Shanghai Clinical Research and Trial Center（上海临床研究与试验中心）
Medical image fusion is the process of registering and combining multiple images from single or multiple imaging modalities to improve the imaging quality and reduce randomness and redundancy in order to increase the clinical applicability of medical images for diagnosis and assessment of medical problems. Multi-modal medical image fusion algorithms and devices have shown notable achievements in improving clinical accuracy of decisions based on medical images. This review article provides a factual listing of methods and summarizes the broad scientific challenges faced in the field of medical …
Every year physicians face an increasing demand of image-based diagnosis from patients, a problem that can be addressed with recent artificial intelligence methods. In this context, we survey works in the area of automatic report generation from medical images, with emphasis on methods using deep neural networks, with respect to: (1) Datasets, (2) Architecture Design, (3) Explainability and (4) Evaluation Metrics. Our survey identifies interesting developments, but also remaining challenges. Among them, the current evaluation of generated reports is especially weak, since it mostly relies on t…
The texture is defined as spatial structure of the intensities of the pixels in an image that is repeated periodically in the whole image or regions, and makes the concept of the image. Texture, color and shape are three main components which are used by human visual system to recognize image contents. In this paper, first of all, efficient and updated texture analysis operators are survived with details. Next, some state-of-the-art methods are survived that use texture analysis in medical applications and disease diagnosis. Finally, different approaches are compared in terms of accuracy, data…
Survey on Adversarial Attack and Defense for Medical Image Analysis: Methods and Challenges
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Guangdong Province Key Laboratory of Information Security Technology（广东省信息安全技术重点实验室）；Division of Life Science, Hong Kong University of Science and Technology（香港科技大学生命科学部）
MBZ University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Inception Institute of Artificial Intelligence（起源人工智能研究院）；Monash University（蒙纳士大学）；Australian National University（澳大利亚国立大学）；Linköping University（林雪平大学）；Agency for Science, Technology and Research（新加坡科技研究局）
Following unprecedented success on the natural language tasks, Transformers have been successfully applied to several computer vision problems, achieving state-of-the-art results and prompting researchers to reconsider the supremacy of convolutional neural networks (CNNs) as {de facto} operators. Capitalizing on these advances in computer vision, the medical imaging field has also witnessed growing interest for Transformers that can capture global context compared to CNNs with local receptive fields. Inspired from this transition, in this survey, we attempt to provide a comprehensive review of…
Jordan University of Science and Technology（约旦科技大学）
The scarcity of high-quality annotated medical imaging datasets is a major problem that collides with machine learning applications in the field of medical imaging analysis and impedes its advancement. Self-supervised learning is a recent training paradigm that enables learning robust representations without the need for human annotation which can be considered an effective solution for the scarcity of annotated medical data. This article reviews the state-of-the-art research directions in self-supervised learning approaches for image data with a concentration on their applications in the fiel…
Implicit Neural Representation in Medical Imaging: A Comparative Survey
Iran University of Science and Technology（伊朗科技大学）；South Dakota State University（南达科他州立大学）；RWTH Aachen University（亚琛工业大学）；University of Regensburg（雷根斯堡大学）；Fraunhofer Institute for Digital Medicine MEVIS（弗劳恩霍夫数字医学MEVIS研究所）
Self-eXplainable AI for Medical Image Analysis: A Survey and New Outlooks
Hong Kong University of Science and Technology（香港科技大学）；Shenzhen MSU-BIT University（深圳北理莫斯科大学）；Harvard University（哈佛大学）；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute（香港科技大学深港协同创新研究院）
Deep Learning Approaches for Medical Imaging Under Varying Degrees of Label Availability: A Comprehensive Survey
University College Dublin（都柏林大学学院）；North China Institute of Aerospace Engineering（北华航天工业学院）；Hithink RoyalFlush Information Network Co., Ltd（同花顺信息网络股份有限公司）
Image Sciences Institute, University Medical Center Utrecht（乌得勒支大学医学中心影像科学研究所）；Utrecht University（乌得勒支大学）
With an increase in deep learning-based methods, the call for explainability of such methods grows, especially in high-stakes decision making areas such as medical image analysis. This survey presents an overview of eXplainable Artificial Intelligence (XAI) used in deep learning-based medical image analysis. A framework of XAI criteria is introduced to classify deep learning-based medical image analysis methods. Papers on XAI techniques in medical image analysis are then surveyed and categorized according to the framework and according to anatomical location. The paper concludes with an outloo…
Foundational Models in Medical Imaging: A Comprehensive Survey and Future Vision
South Dakota State University（南达科他州立大学）；RWTH Aachen University（亚琛工业大学）；University of Kentucky（肯塔基大学）；University of Regensburg（雷根斯堡大学）；Iran University of Science and Technology（伊朗科技大学）；Imperial College London（伦敦帝国理工学院）
Medical Image Registration and Its Application in Retinal Images: A Review
Southern University of Science and Technology（南方科技大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Singapore Eye Research Institute（新加坡眼科研究所）
INESC TEC（葡萄牙系统与计算机工程研究所）；University of Porto（波尔图大学）
The increasing popularity of attention mechanisms in deep learning algorithms for computer vision and natural language processing made these models attractive to other research domains. In healthcare, there is a strong need for tools that may improve the routines of the clinicians and the patients. Naturally, the use of attention-based algorithms for medical applications occurred smoothly. However, being healthcare a domain that depends on high-stake decisions, the scientific community must ponder if these high-performing algorithms fit the needs of medical applications. With this motto, this …
University of Pittsburgh（匹兹堡大学）；Carnegie Mellon University（卡内基梅隆大学）；Stanford University（斯坦福大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
Computer vision systems require large amounts of manually annotated data to properly learn challenging visual concepts. Crowdsourcing platforms offer an inexpensive method to capture human knowledge and understanding, for a vast number of visual perception tasks. In this survey, we describe the types of annotations computer vision researchers have collected using crowdsourcing, and how they have ensured that this data is of high quality while annotation effort is minimized. We begin by discussing data collection on both classic (e.g., object recognition) and recent (e.g., visual story-telling)…
J. J. Strossmayer University of Osijek（奥西耶克J.J.斯特罗斯马耶大学）；Ghent University（根特大学）；imec（比利时微电子研究中心）
Atrial fibrillation (AF) is a cardiovascular disease identified as one of the main risk factors for stroke. The majority of strokes due to AF are caused by clots originating in the left atrial appendage (LAA). LAA occlusion is an effective procedure for reducing stroke risk. Planning the procedure using pre-procedural imaging and analysis has shown benefits. The analysis is commonly done by manually segmenting the appendage on 2D slices. Automatic LAA segmentation methods could save an expert's time and provide insightful 3D visualizations and accurate automatic measurements to aid in medical …
A Comprehensive Survey of Mamba Architectures for Medical Image Analysis: Classification, Segmentation, Restoration and Beyond
Indian Institute of Technology Indore, India；R.M.D. Engineering College, Kavaraipettai, India；Jio Platforms Limited, India；Birla Institute of Technology \& Science Pilani, India
The availability of training data for supervision is a frequently encountered bottleneck of medical image analysis methods. While typically established by a clinical expert rater, the increase in acquired imaging data renders traditional pixel-wise segmentations less feasible. In this paper, we examine the use of a crowdsourcing platform for the distribution of super-pixel weak annotation tasks and collect such annotations from a crowd of non-expert raters. The crowd annotations are subsequently used for training a fully convolutional neural network to address the problem of fetal brain segmen…
Fully automatic deep learning has become the state-of-the-art technique for many tasks including image acquisition, analysis and interpretation, and for the extraction of clinically useful information for computer-aided detection, diagnosis, treatment planning, intervention and therapy. However, the unique challenges posed by medical image analysis suggest that retaining a human end user in any deep learning enabled system will be beneficial. In this review we investigate the role that humans might play in the development and deployment of deep learning enabled diagnostic applications and focu…
Survey on deep learning in multimodal medical imaging for cancer detection
Massey University（梅西大学）；Nantong University（南通大学）；Zhejiang Engineering Research Center of Intelligent Medicine（浙江省智能医学工程研究中心）；Jianpei Technology Co. Ltd（建培科技有限公司）
In recent years, the prevalence of several pulmonary diseases, especially the coronavirus disease 2019 (COVID-19) pandemic, has attracted worldwide attention. These diseases can be effectively diagnosed and treated with the help of lung imaging. With the development of deep learning technology and the emergence of many public medical image datasets, the diagnosis of lung diseases via medical imaging has been further improved. This article reviews pulmonary CT and X-ray image detection and classification in the last decade. It also provides an overview of the detection of lung nodules, pneumoni…
Linköping University（林雪平大学）；Center for Medical Image Science and Visualization, Linköping University（林雪平大学医学影像科学与可视化中心）；Sectra AB
Artificial intelligence (AI) has shown great promise for diagnostic imaging assessments. However, the application of AI to support medical diagnostics in clinical routine comes with many challenges. The algorithms should have high prediction accuracy but also be transparent, understandable and reliable. Thus, explainable artificial intelligence (XAI) is highly relevant for this domain. We present a survey on XAI within digital pathology, a medical imaging sub-discipline with particular characteristics and needs. The review includes several contributions. Firstly, we give a thorough overview of…
ShanghaiTech University（上海科技大学）；University of Chinese Academy of Sciences（中国科学院大学）
Machine learning has been developed dramatically and witnessed a lot of applications in various fields over the past few years. This boom originated in 2009, when a new model emerged, that is, the deep artificial neural network, which began to surpass other established mature models on some important benchmarks. Later, it was widely used in academia and industry. Ranging from image analysis to natural language processing, it fully exerted its magic and now become the state-of-the-art machine learning models. Deep neural networks have great potential in medical imaging technology, medical data …
Computation-Efficient Era: A Comprehensive Survey of State Space Models in Medical Image Analysis
School of Biomedical Engineering（生物医学工程学院）；University of British Columbia（不列颠哥伦比亚大学）；Tarbiat Modares University（塔比亚特·莫达雷斯大学）；University of Regensburg（雷根斯堡大学）；South Dakota State University（南达科他州立大学）；RWTH Aachen University（亚琛工业大学）；Lunit Inc.（Lunit公司）；Northwestern University（西北大学）；Fraunhofer Institute for Digital Medicine（弗劳恩霍夫数字医学研究院）
Development of Skip Connection in Deep Neural Networks for Computer Vision and Medical Image Analysis: A Survey
School of Computer Science and Engineering, Hubei Key Laboratory of Intelligent Robot, Wuhan Institute of Technology（计算机科学与工程学院、智能机器人湖北省重点实验室、武汉理工大学）；School of Computer Science, Wuhan University（计算机科学学院、武汉大学）
Queensland University of Technology (QUT)（昆士兰科技大学）；Commonwealth Scientific and Industrial Research Organisation (CSIRO)（联邦科学与工业研究组织）；Deakin University（迪肯大学）
We present a medical crowdsourcing visual analytics platform called C{$^2$}A to visualize, classify and filter crowdsourced clinical data. More specifically, C$^2$A is used to build consensus on a clinical diagnosis by visualizing crowd responses and filtering out anomalous activity. Crowdsourcing medical applications have recently shown promise where the non-expert users (the crowd) were able to achieve accuracy similar to the medical experts. This has the potential to reduce interpretation/reading time and possibly improve accuracy by building a consensus on the findings beforehand and letti…
Prompt Mechanisms in Medical Imaging: A Comprehensive Survey
Faculty of Applied Sciences, Macao Polytechnic University（应用科学学院，澳门理工学院）；Netherlands Cancer Institute, Department of Radiology（荷兰癌症研究所，放射科）；Radboud University Medical Centre, Department of Radiology and Nuclear Medicine（拉德堡德大学医学中心，放射科和核医学科）；College of Aerospace Science and Engineering, National University of Defense Technology（航空航天科学与工程学院，国防科技大学）；Medical Department of Breast Cancer, Hunan Cancer Hospital（乳腺癌医学部，湖南癌症医院）；the Affiliated Cancer Hospital of Xiangya School of Medicine, Central South University（湘雅医学院附属肿瘤医院，中南大学）；Faculty of Biomedical Engineering, Eindhoven University of Technology（生物医学工程学院，埃因霍温理工大学）；Laboratory of Advanced Theranostic Materials and Technology, University of Chinese Academy of Sciences（先进诊疗材料与技术实验室，中国科学院大学）
The Hong Kong University of Science and Technology（香港科技大学）；The Chinese University of Hong Kong（香港中文大学）；Southern University of Science and Technology（南方科技大学）
Computational cytology is a critical, rapid-developing, yet challenging topic in the field of medical image computing which analyzes the digitized cytology image by computer-aided technologies for cancer screening. Recently, an increasing number of deep learning (DL) algorithms have made significant progress in medical image analysis, leading to the boosting publications of cytological studies. To investigate the advanced methods and comprehensive applications, we survey more than 120 publications of DL-based cytology image analysis in this article. We first introduce various deep learning met…
SINTEF Digital（SINTEF数字研究院）；KTH Royal Institute of Technology（瑞典皇家理工学院）；Northwestern University（西北大学）
Due to the current COVID-19 virus, which has already been declared a pandemic by the World Health Organization (WHO), we are witnessing the greatest pandemic of the decade. Millions of people are being infected, resulting in thousands of deaths every day across the globe. Even it was difficult for the best healthcare-providing countries could not handle the pandemic because of the strain of treating thousands of patients at a time. The count of infections and deaths is increasing at an alarming rate because of the spread of the virus. We believe that innovative technologies could help reduce p…
Medical image segmentation, the task of partitioning an image into meaningful parts, is an important step toward automating medical image analysis and is at the crux of a variety of medical imaging applications, such as computer aided diagnosis, therapy planning and delivery, and computer aided interventions. However, the existence of noise, low contrast and objects' complexity in medical images are critical obstacles that stand in the way of achieving an ideal segmentation system. Incorporating prior knowledge into image segmentation algorithms has proven useful for obtaining more accurate an…
AI-Enhanced Virtual Reality in Medicine: A Comprehensive Survey
School of Public Health, Zhejiang University（浙江大学公共卫生学院）；The Second Affiliated Hospital, Zhejiang University School of Medicine（浙江大学医学院附属第二医院）；Institute of Wenzhou, Zhejiang University（浙江大学温州研究院）；School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）理工学院）；University of Notre Dame（圣母大学）
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Machine learning techniques used in computer-aided medical image analysis usually suffer from the domain shift problem caused by different distributions between source/reference data and target data. As a promising solution, domain adaptation has attracted considerable attention in recent years. The aim of this paper is to survey the recent advances of domain adaptation methods in medical image analysis. We first present the motivation of introducing domain adaptation techniques to tackle domain heterogeneity issues for medical image analysis. Then we provide a review of recent domain adaptati…
National Institute of Technology Karnataka（卡纳塔克邦国家技术研究所）
Computer-aided medical image analysis plays a significant role in assisting medical practitioners for expert clinical diagnosis and deciding the optimal treatment plan. At present, convolutional neural networks (CNN) are the preferred choice for medical image analysis. In addition, with the rapid advancements in three-dimensional (3D) imaging systems and the availability of excellent hardware and software support to process large volumes of data, 3D deep learning methods are gaining popularity in medical image analysis. Here, we present an extensive review of the recently evolved 3D deep learn…
NepAl Applied Mathematics and Informatics Institute for research (NAAMII)（尼泊尔应用数学与信息学研究所（NAAMII））；Rochester Institute of Technology（罗切斯特理工学院）；University of Aberdeen（阿伯丁大学）
Shaanxi University of Science and Technology（陕西科技大学）；Lanzhou Jiaotong University（兰州交通大学）；Brunel University London（伦敦布鲁内尔大学）
Deep learning has been widely used for medical image segmentation and a large number of papers has been presented recording the success of deep learning in the field. In this paper, we present a comprehensive thematic survey on medical image segmentation using deep learning techniques. This paper makes two original contributions. Firstly, compared to traditional surveys that directly divide literatures of deep learning on medical image segmentation into many groups and introduce literatures in detail for each group, we classify currently popular literatures according to a multi-level structure…
A Survey of Medical Vision-and-Language Applications and Their Techniques
Australian Institute for Machine Learning（澳大利亚机器学习研究所）；University of Adelaide（阿德莱德大学）；School of Software Engineering（软件工程学院）；South China University of Technology（华南理工大学）；South Australia Medical Imaging（南澳大利亚医学影像中心）；School of Computer Science & Engineering（计算机科学与工程学院）；Northwestern Polytechnical University（西北工业大学）
