We collected 32 public datasets, of which 28 for medical imaging and 4 for natural images, to conduct study. The images of these datasets are captured by different cameras, thus vary from each other in modality, frame size and capacity. For data accessibility, we also provide the websites of most datasets and hope this will help the readers reach the datasets.
Image Sciences Institute, University Medical Center Utrecht（乌得勒支大学医学中心影像科学研究所）；Utrecht University（乌得勒支大学）
With an increase in deep learning-based methods, the call for explainability of such methods grows, especially in high-stakes decision making areas such as medical image analysis. This survey presents an overview of eXplainable Artificial Intelligence (XAI) used in deep learning-based medical image analysis. A framework of XAI criteria is introduced to classify deep learning-based medical image analysis methods. Papers on XAI techniques in medical image analysis are then surveyed and categorized according to the framework and according to anatomical location. The paper concludes with an outloo…
Emory University（埃默里大学）；Winship Cancer Institute（温希普癌症研究所）；Emory University School of Medicine（埃默里大学医学院）
Medical imaging is widely used in cancer diagnosis and treatment, and artificial intelligence (AI) has achieved tremendous success in various tasks of medical image analysis. This paper reviews AI-based tumor subregion analysis in medical imaging. We summarize the latest AI-based methods for tumor subregion analysis and their applications. Specifically, we categorize the AI-based methods by training strategy: supervised and unsupervised. A detailed review of each category is presented, highlighting important contributions and achievements. Specific challenges and potential AI applications in t…
Xidian University（西安电子科技大学）；The University of Notre Dame（圣母大学）；Shanghai Broadband Network Center（上海宽带网络中心）；The Third Research Institute of The Ministry of Public Security（公安部第三研究所）；Murdoch University（莫道克大学）；The University of Western Australia（西澳大学）
The astounding success made by artificial intelligence (AI) in healthcare and other fields proves that AI can achieve human-like performance. However, success always comes with challenges. Deep learning algorithms are data-dependent and require large datasets for training. The lack of data in the medical imaging field creates a bottleneck for the application of deep learning to medical image analysis. Medical image acquisition, annotation, and analysis are costly, and their usage is constrained by ethical restrictions. They also require many resources, such as human expertise and funding. That…
Thomas Jefferson University Hospital（托马斯·杰斐逊大学医院）；Sidney Kimmel Jefferson Medical College（西德尼·金梅尔杰斐逊医学院）
The discussions around Artificial Intelligence (AI) and medical imaging are centered around the success of deep learning algorithms. As new algorithms enter the market, it is important for practicing radiologists to understand the pitfalls of various AI algorithms. This entails having a basic understanding of how algorithms are developed, the kind of data they are trained on, and the settings in which they will be deployed. As with all new technologies, use of AI should be preceded by a fundamental understanding of the risks and benefits to those it is intended to help. This case-based review …
Empowering Medical Imaging with Artificial Intelligence: A Review of Machine Learning Approaches for the Detection, and Segmentation of COVID-19 Using Radiographic and Tomographic Images
Stanford University（斯坦福大学）；Columbia University（哥伦比亚大学）；Stanford Artificial Intelligence in Medicine Center（斯坦福医学人工智能中心）
Advances in computing power, deep learning architectures, and expert labelled datasets have spurred the development of medical imaging artificial intelligence systems that rival clinical experts in a variety of scenarios. The National Institutes of Health in 2018 identified key focus areas for the future of artificial intelligence in medical imaging, creating a foundational roadmap for research in image acquisition, algorithms, data standardization, and translatable clinical decision support systems. Among the key issues raised in the report: data availability, need for novel computing archite…
Aalto University（阿尔托大学）；KTH Royal Institute of Technology（瑞典皇家理工学院）
In the present paper we present the potential of Explainable Artificial Intelligence methods for decision-support in medical image analysis scenarios. With three types of explainable methods applied to the same medical image data set our aim was to improve the comprehensibility of the decisions provided by the Convolutional Neural Network (CNN). The visual explanations were provided on in-vivo gastral images obtained from a Video capsule endoscopy (VCE), with the goal of increasing the health professionals' trust in the black box predictions. We implemented two post-hoc interpretable machine l…
This review presents a comprehensive study on the role of GANs in addressing the challenges related to COVID-19 data scarcity and diagnosis. It is the first review that summarizes the different GANs methods and the lungs images datasets for COVID-19. It attempts to answer the questions related to applications of GANs, popular GAN architectures, frequently used image modalities, and the availability of source code. This review included 57 full-text studies that reported the use of GANs for different applications in COVID-19 lungs images data. Most of the studies (n=42) used GANs for data augmen…
Mila, Quebec Artificial Intelligence Institute（Mila，魁北克人工智能研究所）；University of Montreal（蒙特利尔大学）；Department of Mathematics and Computer Science, Fontbonne University（方特博恩大学数学与计算机科学系）；Faculty of Medicine, University of Montreal（蒙特利尔大学医学院）
This paper describes the initial COVID-19 open image data collection. It was created by assembling medical images from websites and publications and currently contains 123 frontal view X-rays.
FUTURE-AI: Guiding Principles and Consensus Recommendations for Trustworthy Artificial Intelligence in Medical Imaging
Barcelona Artificial Intelligence in Medicine Lab (BCN-AIM)（巴塞罗那医学人工智能实验室（BCN-AIM））；Maggioli SPA（马焦利股份公司）；Fundació TIC Salut i Social（信息与通信技术社会基金会）；Generalitat de Catalunya（加泰罗尼亚政府）；Hospital Universitario y Politécnico La Fe（拉费大学及综合医院）；Health Research Institute（健康研究所）；Foundation for Research and Technology（研究与技术基金会）；National Research Council（国家研究委员会）；Maastricht University（马斯特里赫特大学）
Peng Cheng Laboratory（鹏城实验室）；Harbin Institute of Technology(Shenzhen)（哈尔滨工业大学（深圳））；National University of Defense Technology（国防科技大学）；Peking University（北京大学）
In this paper, we present OpenMedIA, an open-source toolbox library containing a rich set of deep learning methods for medical image analysis under heterogeneous Artificial Intelligence (AI) computing platforms. Various medical image analysis methods, including 2D/3D medical image classification, segmentation, localisation, and detection, have been included in the toolbox with PyTorch and/or MindSpore implementations under heterogeneous NVIDIA and Huawei Ascend computing systems. To our best knowledge, OpenMedIA is the first open-source algorithm library providing compared PyTorch and MindSpor…
All-in-one platform for AI R&D in medical imaging, encompassing data collection, selection, annotation, and pre-processing
Callisto Inc（卡利斯托公司）；The University of Tokyo（东京大学）；St. Marianna University School of Medicine（圣玛丽安娜医科大学医学院）；Harada Academy（原田学园）；St. Marianna University Yokohama Seibu Hospital（圣玛丽安娜大学横滨西部医院）
Generative Artificial Intelligence in Medical Imaging: Foundations, Progress, and Clinical Translation
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Macao Polytechnic University（澳门理工大学）
Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能医疗科技有限公司）；Shanghai University（上海大学）；United Imaging Intelligence（联影智能）；Shanghai Jiao Tong University（上海交通大学）；Beihang University（北京航空航天大学）；Medical School of Nanjing University（南京大学医学院）；National Institute of Healthcare Data Science at Nanjing University（南京大学国家医疗数据科学研究院）
(This paper was submitted as an invited paper to IEEE Reviews in Biomedical Engineering on April 6, 2020.) The pandemic of coronavirus disease 2019 (COVID-19) is spreading all over the world. Medical imaging such as X-ray and computed tomography (CT) plays an essential role in the global fight against COVID-19, whereas the recently emerging artificial intelligence (AI) technologies further strengthen the power of the imaging tools and help medical specialists. We hereby review the rapid responses in the community of medical imaging (empowered by AI) toward COVID-19. For example, AI-empowered i…
Medical University of Graz（格拉茨医科大学）；Computer Algorithms for Medicine Laboratory（医学计算机算法实验室）；Graz University of Technology（格拉茨工业大学）
Image-based algorithmic software segmentation is an increasingly important topic in many medical fields. Algorithmic segmentation is used for medical three-dimensional visualization, diagnosis or treatment support, especially in complex medical cases. However, accessible medical databases are limited, and valid medical ground truth databases for the evaluation of algorithms are rare and usually comprise only a few images. Inaccuracy or invalidity of medical ground truth data and image-based artefacts also limit the creation of such databases, which is especially relevant for CT data sets of th…
Imperial College London（帝国理工学院）；University of Edinburgh（爱丁堡大学）；Technical University of Munich（慕尼黑工业大学）
Artificial intelligence (AI) and Machine Learning (ML) have shown great potential in improving the medical imaging workflow, from image acquisition and reconstruction to disease diagnosis and treatment. Particularly, in recent years, there has been a significant growth in the use of AI and ML algorithms, especially Deep Learning (DL) based methods, for medical image reconstruction. DL techniques have shown to be competitive and often superior over conventional reconstruction methods in terms of both reconstruction quality and computational efficiency. The use of DL-based image reconstruction a…
Linköping University（林雪平大学）；Center for Medical Image Science and Visualization, Linköping University（林雪平大学医学影像科学与可视化中心）；Sectra AB
Artificial intelligence (AI) has shown great promise for diagnostic imaging assessments. However, the application of AI to support medical diagnostics in clinical routine comes with many challenges. The algorithms should have high prediction accuracy but also be transparent, understandable and reliable. Thus, explainable artificial intelligence (XAI) is highly relevant for this domain. We present a survey on XAI within digital pathology, a medical imaging sub-discipline with particular characteristics and needs. The review includes several contributions. Firstly, we give a thorough overview of…
Medical Artificial Intelligence for Early Detection of Lung Cancer: A Survey
College of Computer Science and Artificial Intelligence, Southwest Minzu University, Chengdu, 610225, China（计算机科学与人工智能学院，西南民族大学）；The Australian National University, Canberra ACT 2601, Australia（澳大利亚国立大学）；School of Computer Science and Technology, Beijing Jiaotong University, Beijing, China（计算机科学与技术学校，北京交通大学）；Hunan university of technology, Zhuzhou, 410072, China（湖南理工大学）；University of Adelaide, Adelaide, South Australia 5005, Australia（阿德莱德大学）；La Trobe University, Melbourne, Victoria 3086, Australia（拉筹伯大学）
MIT（麻省理工学院）；Massachusetts General Hospital（麻省总医院）；HMS（哈佛医学院）；Google（谷歌）；Cornell University（康奈尔大学）
We present an algorithm for creating high resolution anatomically plausible images consistent with acquired clinical brain MRI scans with large inter-slice spacing. Although large data sets of clinical images contain a wealth of information, time constraints during acquisition result in sparse scans that fail to capture much of the anatomy. These characteristics often render computational analysis impractical as many image analysis algorithms tend to fail when applied to such images. Highly specialized algorithms that explicitly handle sparse slice spacing do not generalize well across problem…
This work provides a strong baseline for the problem of multi-source multi-target domain adaptation and generalization in medical imaging. Using a diverse collection of ten chest X-ray datasets, we empirically demonstrate the benefits of training medical imaging deep learning models on varied patient populations for generalization to out-of-sample domains.
Towards a Transparent and Interpretable AI Model for Medical Image Classifications
School of Artificial Intelligence, Guilin University of Electronic Technology（桂林电子科技大学人工智能学院）；The Laboratory for Imagery, Vision and Artificial Intelligence, Ecole de Technologie Superieure（图像、视觉与人工智能实验室，超技术学院）；College of Medicine, Taibah University（塔伊巴大学医学院）；Prince Mohammed Bin Abdulaziz Hospital, Ministry of National Guard Health Affairs（穆罕默德·本·阿卜杜勒阿齐兹医院，国家守卫健康事务部）
A Review on the Integration of Artificial Intelligence and Medical Imaging in IVF Ovarian Stimulation
Danube Private University (DPU)（多瑙河私立大学）；Kinderwunschzentrum an der Wien（维也纳生育中心）；Research Center for Clinical AI-Research in Omics and Medical Data Science (CAROM)（组学与医学数据科学临床人工智能研究中心）；Austrian Center for Medical Innovation and Technology (ACMIT)（奥地利医学创新与技术中心）；Medical University of Vienna (MUV)（维也纳医科大学）；Laboratory for Life Sciences and Technology (LiST)（生命科学与技术实验室）；Technion – Israel Institute of Technology（以色列理工学院）
Sun Yat-sen University（中山大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Xi’an Jiaotong University（西安交通大学）；Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College（中国医学科学院北京协和医学院北京协和医院）；University of Portsmouth（朴茨茅斯大学）；Rensselaer Polytechnic Institute（伦斯勒理工学院）；Qingdao Academy of Intelligent Industries（青岛智能产业技术研究院）
There has been much progress in data-driven artificial intelligence technology for medical image analysis in the last decades. However, it still remains challenging due to its distinctive complexity of acquiring and annotating image data, extracting medical domain knowledge, and explaining the diagnostic decision for medical image analysis. In this paper, we propose a data-knowledge-driven framework termed as Parallel Medical Imaging (PMI) for intelligent medical image analysis based on the methodology of interactive ACP-based parallel intelligence. In the PMI framework, computational experime…
Multimodal Human-AI Synergy for Medical Imaging Quality Control: A Hybrid Intelligence Framework with Adaptive Dataset Curation and Closed-Loop Evaluation
Zhongda Hospital（中大医院）；School of Medicine, Southeast University（东南大学医学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
College of Science and Engineering, Hamad Bin Khalifa University（哈马德·本·哈利法大学科学与工程学院）；College of Health and Life Sciences, Hamad Bin Khalifa University（哈马德·本·哈利法大学健康与生命科学学院）
Healthcare data are inherently multimodal, including electronic health records (EHR), medical images, and multi-omics data. Combining these multimodal data sources contributes to a better understanding of human health and provides optimal personalized healthcare. Advances in artificial intelligence (AI) technologies, particularly machine learning (ML), enable the fusion of these different data modalities to provide multimodal insights. To this end, in this scoping review, we focus on synthesizing and analyzing the literature that uses AI techniques to fuse multimodal medical data for different…
We present MedMNIST, a collection of 10 pre-processed medical open datasets. MedMNIST is standardized to perform classification tasks on lightweight 28x28 images, which requires no background knowledge. Covering the primary data modalities in medical image analysis, it is diverse on data scale (from 100 to 100,000) and tasks (binary/multi-class, ordinal regression and multi-label). MedMNIST could be used for educational purpose, rapid prototyping, multi-modal machine learning or AutoML in medical image analysis. Moreover, MedMNIST Classification Decathlon is designed to benchmark AutoML algori…
PETWB-REP: A Multi-Cancer Whole-Body FDG PET/CT and Radiology Report Dataset for Medical Imaging Research
PET Center, Huashan Hospital, Fudan University（复旦大学附属华山医院PET中心）；Shanghai Academy of Artificial Intelligence for Science（上海科学智能研究院）；Institute of Science and Technology for Brain-inspired Intelligence, Fudan University（复旦大学类脑智能科学与技术研究院）；Shanghai Universal Medical Imaging Diagnostic Center（上海环球医学影像诊断中心）；Human Phenome Institute, Fudan University（复旦大学人类表型组研究院）；School of Data Science, Fudan University（复旦大学数据科学学院）
Every year physicians face an increasing demand of image-based diagnosis from patients, a problem that can be addressed with recent artificial intelligence methods. In this context, we survey works in the area of automatic report generation from medical images, with emphasis on methods using deep neural networks, with respect to: (1) Datasets, (2) Architecture Design, (3) Explainability and (4) Evaluation Metrics. Our survey identifies interesting developments, but also remaining challenges. Among them, the current evaluation of generated reports is especially weak, since it mostly relies on t…
In order to be able to use artificial intelligence (AI) in medicine without scepticism and to recognise and assess its growing potential, a basic understanding of this topic is necessary among current and future medical staff. Under the premise of "trust through understanding", we developed an innovative online course as a learning opportunity within the framework of the German KI Campus (AI campus) project, which is a self-guided course that teaches the basics of AI for the analysis of medical image data. The main goal is to provide a learning environment for a sufficient understanding of AI …
Peking University（北京大学）；Yizhun Medical AI Co., Ltd（医准医疗人工智能有限公司）
Applying artificial intelligence techniques in medical imaging is one of the most promising areas in medicine. However, most of the recent success in this area highly relies on large amounts of carefully annotated data, whereas annotating medical images is a costly process. In this paper, we propose a novel method, called FocalMix, which, to the best of our knowledge, is the first to leverage recent advances in semi-supervised learning (SSL) for 3D medical image detection. We conducted extensive experiments on two widely used datasets for lung nodule detection, LUNA16 and NLST. Results show th…
Brighton and Sussex University Hospitals, NHS Trust（布莱顿和苏塞克斯大学医院国民保健服务信托）；Institute of Theoretical Physics, Chinese Academy of Sciences（中国科学院理论物理研究所）；Alan Turing Institute（艾伦·图灵研究所）；University of Sussex（萨塞克斯大学）
In this paper we present our work to improve access to diagnosis in remote areas where good quality medical services may be lacking. We develop new Machine Learning methodologies for deployment onto mobile devices to help the early diagnosis of a number of life-threatening conditions using X-ray images. By using the latest developments in fast and portable Artificial Intelligence environments, we develop a smartphone app using an Artificial Neural Network to assist physicians in their diagnostic.
Despite significant advances in artificial intelligence (AI) for computer vision, its application in medical imaging has been limited by the burden and limits of expert-generated labels. We used images from optical coherence tomography angiography (OCTA), a relatively new imaging modality that measures perfusion of the retinal vasculature, to train an AI algorithm to generate vasculature maps from standard structural optical coherence tomography (OCT) images of the same retinae, both exceeding the ability and bypassing the need for expert labeling. Deep learning was able to infer perfusion of …
Machine learning and artificial intelligence have shown remarkable performance in accelerated magnetic resonance imaging (MRI). Cloud computing technologies have great advantages in building an easily accessible platform to deploy advanced algorithms. In this work, we develop an open-access, easy-to-use and high-performance medical intelligence cloud computing platform (XCloud-pFISTA) to reconstruct MRI images from undersampled k-space data. Two state-of-the-art approaches of the Projected Fast Iterative Soft-Thresholding Algorithm (pFISTA) family have been successfully implemented on the clou…
Towards Collective Intelligence: Uncertainty-aware SAM Adaptation for Ambiguous Medical Image Segmentation
Department of Computer Science, The University of Alabama at Birmingham（阿拉巴马大学伯明翰分校计算机科学系）；Doctoral Training Centre, University of Oxford（牛津大学博士培训中心）；Department of Biomedical Engineering and Department of Electrical and Computer Engineering, National University of Singapore（新加坡国立大学生物医学工程系和电气与计算机工程系）；Computational Biology Department, Carnegie Mellon University（卡内基梅隆大学计算生物学系）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Artificial intelligence (AI)-based methods are showing promise in multiple medical-imaging applications. Thus, there is substantial interest in clinical translation of these methods, requiring in turn, that they be evaluated rigorously. In this paper, our goal is to lay out a framework for objective task-based evaluation of AI methods. We will also provide a list of tools available in the literature to conduct this evaluation. Further, we outline the important role of physicians in conducting these evaluation studies. The examples in this paper will be proposed in the context of PET with a foc…
Ontology-based knowledge representation for bone disease diagnosis: a foundation for safe and sustainable medical artificial intelligence systems
University of Science, VNUHCM（胡志明市国家大学科学大学）；Viet Nam National University, Ho Chi Minh City（越南国家大学胡志明市分校）；Ho Chi Minh City Hospital for Traumatology and Orthopedics（胡志明市创伤与骨科医院）
Building structures detection and information about these buildings in aerial images is an important solution for city planning and management, land use analysis. It can be the center piece to answer important questions such as planning evacuation routes in case of an earthquake, flood management, etc. These applications rely on being able to accurately retrieve up-to-date information. Being able to accurately detect buildings in a bounding box centered on a specific latitude-longitude value can help greatly. The key challenge is to be able to detect buildings which can be commercial, industri…
School of Engineering, Lakehead University（湖首大学工程学院）
Alzheimer's Disease (AD) is a severe brain disorder, destroying memories and brain functions. AD causes chronically, progressively, and irreversibly cognitive declination and brain damages. The reliable and effective evaluation of early dementia has become essential research with medical imaging technologies and computer-aided algorithms. This trend has moved to modern Artificial Intelligence (AI) technologies motivated by deeplearning success in image classification and natural language processing. The purpose of this review is to provide an overview of the latest research involving deep-lear…
University of Augsburg（奥格斯堡大学）；University Hospital Augsburg（奥格斯堡大学医院）
In the last decade, research on artificial intelligence has seen rapid growth with deep learning models, especially in the field of medical image segmentation. Various studies demonstrated that these models have powerful prediction capabilities and achieved similar results as clinicians. However, recent studies revealed that the evaluation in image segmentation studies lacks reliable model performance assessment and showed statistical bias by incorrect metric implementation or usage. Thus, this work provides an overview and interpretation guide on the following metrics for medical image segmen…
