Data quality is the key factor for the development of trustworthy AI in healthcare. A large volume of curated datasets with controlled confounding factors can help improve the accuracy, robustness and privacy of downstream AI algorithms. However, access to good quality datasets is limited by the technical difficulty of data acquisition and large-scale sharing of healthcare data is hindered by strict ethical restrictions. Data synthesis algorithms, which generate data with a similar distribution as real clinical data, can serve as a potential solution to address the scarcity of good quality data during the development of trustworthy AI. However, state-of-the-art data synthesis algorithms, especially deep learning algorithms, focus more on imaging data while neglecting the synthesis of non-imaging healthcare data, including clinical measurements, medical signals and waveforms, and electronic healthcare records (EHRs). Thus, in this paper, we will review the synthesis algorithms, particularly for non-imaging medical data, with the aim of providing trustworthy AI in this domain. This tutorial-styled review paper will provide comprehensive descriptions of non-imaging medical data synthesis on aspects including algorithms, evaluations, limitations and future research directions.
FUTURE-AI: Guiding Principles and Consensus Recommendations for Trustworthy Artificial Intelligence in Medical Imaging
Barcelona Artificial Intelligence in Medicine Lab (BCN-AIM)（巴塞罗那医学人工智能实验室（BCN-AIM））；Maggioli SPA（马焦利股份公司）；Fundació TIC Salut i Social（信息与通信技术社会基金会）；Generalitat de Catalunya（加泰罗尼亚政府）；Hospital Universitario y Politécnico La Fe（拉费大学及综合医院）；Health Research Institute（健康研究所）；Foundation for Research and Technology（研究与技术基金会）；National Research Council（国家研究委员会）；Maastricht University（马斯特里赫特大学）
University of Waterloo（滑铁卢大学）；National Research Council Canada（加拿大国家研究委员会）
Computer vision and machine learning are playing an increasingly important role in computer-assisted diagnosis; however, the application of deep learning to medical imaging has challenges in data availability and data imbalance, and it is especially important that models for medical imaging are built to be trustworthy. Therefore, we propose TRUDLMIA, a trustworthy deep learning framework for medical image analysis, which adopts a modular design, leverages self-supervised pre-training, and utilizes a novel surrogate loss function. Experimental evaluations indicate that models generated from the…
Xidian University（西安电子科技大学）；Northwestern Polytechnical University（西北工业大学）；Hefei Comprehensive National Science Center（合肥综合性国家科学中心）；Nankai University（南开大学）；City University of Hong Kong（香港城市大学）
Due to the constraints of the imaging device and high cost in operation time, computer tomography (CT) scans are usually acquired with low intra-slice resolution. Improving the intra-slice resolution is beneficial to the disease diagnosis for both human experts and computer-aided systems. To this end, this paper builds a novel medical slice synthesis to increase the between-slice resolution. Considering that the ground-truth intermediate medical slices are always absent in clinical practice, we introduce the incremental cross-view mutual distillation strategy to accomplish this task in the sel…
Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；Inserm（法国国家健康与医学研究院）；Pixyl Research and Development Laboratory（Pixyl研发实验室）；Inria（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔理工学院）
The full acceptance of Deep Learning (DL) models in the clinical field is rather low with respect to the quantity of high-performing solutions reported in the literature. Particularly, end users are reluctant to rely on the rough predictions of DL models. Uncertainty quantification methods have been proposed in the literature as a potential response to reduce the rough decision provided by the DL black box and thus increase the interpretability and the acceptability of the result by the final user. In this review, we propose an overview of the existing methods to quantify uncertainty associate…
Although machine learning (ML) models of AI achieve high performances in medicine, they are not free of errors. Empowering clinicians to identify incorrect model recommendations is crucial for engendering trust in medical AI. Explainable AI (XAI) aims to address this requirement by clarifying AI reasoning to support the end users. Several studies on biomedical imaging achieved promising results recently. Nevertheless, solutions for models using tabular data are not sufficient to meet the requirements of clinicians yet. This paper proposes a methodology to support clinicians in identifying fail…
Institute for Artificial Intelligence in Medicine, University Hospital Essen（埃森大学医院医学人工智能研究所）；University of Duisburg-Essen（杜伊斯堡-埃森大学）；Cancer Research Center Cologne Essen（科隆-埃森癌症研究中心）
AI/Computing at scale is a difficult problem, especially in a health care setting. We outline the requirements, planning and implementation choices as well as the guiding principles that led to the implementation of our secure research computing enclave, the Essen Medical Computing Platform (EMCP), affiliated with a major German hospital. Compliance, data privacy and usability were the immutable requirements of the system. We will discuss the features of our computing enclave and we will provide our recipe for groups wishing to adopt a similar setup.
Exploration of the Rashomon Set Assists Trustworthy Explanations for Medical Data
University of Warsaw（华沙大学）；Warsaw University of Technology（华沙理工大学）；Medical University of Warsaw（华沙医科大学）；Medical University of Silesia（西里西亚医科大学）；Medical University of Gdańsk（格但斯克医科大学）
University of Waterloo（滑铁卢大学）；National Research Council Canada（加拿大国家研究委员会）
Building AI models with trustworthiness is important especially in regulated areas such as healthcare. In tackling COVID-19, previous work uses convolutional neural networks as the backbone architecture, which has shown to be prone to over-caution and overconfidence in making decisions, rendering them less trustworthy -- a crucial flaw in the context of medical imaging. In this study, we propose a feature learning approach using Vision Transformers, which use an attention-based mechanism, and examine the representation learning capability of Transformers as a new backbone architecture for medi…
Toward Automated Regulatory Decision-Making: Trustworthy Medical Device Risk Classification with Multimodal Transformers and Self-Training
Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford（生物医学工程研究所，工程科学系，牛津大学）；Birmingham Law School, University of Birmingham（伯明翰法学院，伯明翰大学）
A Dempster-Shafer approach to trustworthy AI with application to fetal brain MRI segmentation
University Hospitals Leuven（鲁汶大学医院）；Technical University Munich（慕尼黑工业大学）；University Hospital Zurich（苏黎世大学医院）；University of Zurich（苏黎世大学）；University College London（伦敦大学学院）；University Hospitals Tanta（坦塔大学医院）；Medical University of Vienna（维也纳医科大学）
Regulatory Approval Is Not Enough: Gaps in Trustworthy AI Reporting in FDA-Cleared Medical Devices
University of Leicester（莱斯特大学）；British Heart Foundation (BHF) Leicester Centre of Research Excellence（英国心脏基金会莱斯特卓越研究中心）；Universitat de Barcelona（巴塞罗那大学）；Chalmers University of Technology（查尔姆斯理工大学）；Universidad de Sevilla（塞维利亚大学）；Queen Mary University of London（伦敦玛丽女王大学）；Institució Catalana de Recerca i Estudis Avançats (ICREA)（加泰罗尼亚高级研究所）
University of Copenhagen（哥本哈根大学）；Rigshospitalet（哥本哈根大学国立医院）
Medical imaging plays a vital role in modern diagnostics and treatment. The temporal nature of disease or treatment progression often results in longitudinal data. Due to the cost and potential harm, acquiring large medical datasets necessary for deep learning can be difficult. Medical image synthesis could help mitigate this problem. However, until now, the availability of GANs capable of synthesizing longitudinal volumetric data has been limited. To address this, we use the recent advances in latent space-based image editing to propose a novel joint learning scheme to explicitly embed tempor…
TrustFed: Enabling Trustworthy Medical AI under Data Privacy Constraints
Department of Applied Mechanics（应用力学系）；Indian Institute of Technology Delhi（印度理工学院德里）；Grainger College of Engineering（工程学院）；Nuclear, Plasma & Radiological Engineering Department（核物理与辐射工程系）；National Center for Supercomputing Applications（国家超级计算应用中心）；Yardi School of Artificial Intelligence (ScAI)（Yardi人工智能学院（ScAI））；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Recently, the cycle-consistent generative adversarial networks (CycleGAN) has been widely used for synthesis of multi-domain medical images. The domain-specific nonlinear deformations captured by CycleGAN make the synthesized images difficult to be used for some applications, for example, generating pseudo-CT for PET-MR attenuation correction. This paper presents a deformation-invariant CycleGAN (DicycleGAN) method using deformable convolutional layers and new cycle-consistency losses. Its robustness dealing with data that suffer from domain-specific nonlinear deformations has been evaluated t…
Advancing AI-Powered Medical Image Synthesis: Insights from MedVQA-GI Challenge Using CLIP, Fine-Tuned Stable Diffusion, and Dream-Booth + LoRA
Department of Computer Science, SCMNS School, Morgan State University, Baltimore, Maryland 21251, USA；Electrical \& Computer Engineering Dept., School of Engineering, Morgan State University, Baltimore, Maryland 21251, USA
Unpredictable ML model behavior on unseen data, especially in the health domain, raises serious concerns about its safety as repercussions for mistakes can be fatal. In this paper, we explore the feasibility of using state-of-the-art out-of-distribution detectors for reliable and trustworthy diagnostic predictions. We select publicly available deep learning models relating to various health conditions (e.g., skin cancer, lung sound, and Parkinson's disease) using various input data types (e.g., image, audio, and motion data). We demonstrate that these models show unreasonable predictions on ou…
Mila-Quebec AI Institute（魁北克人工智能研究所Mila）；Dialogue Health Technologies Inc.（Dialogue健康科技公司）
The automation of the medical evidence acquisition and diagnosis process has recently attracted increasing attention in order to reduce the workload of doctors and democratize access to medical care. However, most works proposed in the machine learning literature focus solely on improving the prediction accuracy of a patient's pathology. We argue that this objective is insufficient to ensure doctors' acceptability of such systems. In their initial interaction with patients, doctors do not only focus on identifying the pathology a patient is suffering from; they instead generate a differential …
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Massachusetts Institute of Technology（麻省理工学院）；Univ. Lille（里尔大学）；Inserm（法国国家健康与医学研究院）；CHU Lille（里尔大学医院）
Image synthesis via Generative Adversarial Networks (GANs) of three-dimensional (3D) medical images has great potential that can be extended to many medical applications, such as, image enhancement and disease progression modeling. However, current GAN technologies for 3D medical image synthesis need to be significantly improved to be readily adapted to real-world medical problems. In this paper, we extend the state-of-the-art StyleGAN2 model, which natively works with two-dimensional images, to enable 3D image synthesis. In addition to the image synthesis, we investigate the controllability a…
Exploiting learning algorithms under scarce data regimes is a limitation and a reality of the medical imaging field. In an attempt to mitigate the problem, we propose a data augmentation protocol based on generative adversarial networks. We condition the networks at a pixel-level (segmentation mask) and at a global-level information (acquisition environment or lesion type). Such conditioning provides immediate access to the image-label pairs while controlling global class specific appearance of the synthesized images. To stimulate synthesis of the features relevant for the segmentation task, a…
RAD: Towards Trustworthy Retrieval-Augmented Multi-modal Clinical Diagnosis
College of Computer Science and Artificial Intelligence, Fudan University（复旦大学计算机科学与人工智能学院）；Shanghai AI Laboratory（上海人工智能实验室）；CMIC, Shanghai Jiao Tong University（上海交通大学计算机学院）；School of Artificial Intelligence, Shanghai Jiao Tong University（上海交通大学人工智能学院）；Institute of Artificial Intelligence for Medicine, Shanghai Jiao Tong University（上海交通大学医学人工智能研究所）
University of Bourgogne Franche-Comte（勃艮第-弗朗什孔泰大学）；CASIS inc.（CASIS公司）；University of Sherbrooke（舍布鲁克大学）；University Hospital of Dijon（第戎大学医院）
Generative Adversarial Networks (GANs) have become increasingly powerful, generating mind-blowing photorealistic images that mimic the content of datasets they were trained to replicate. One recurrent theme in medical imaging is whether GANs can also be effective at generating workable medical data as they are for generating realistic RGB images. In this paper, we perform a multi-GAN and multi-application study to gauge the benefits of GANs in medical imaging. We tested various GAN architectures from basic DCGAN to more sophisticated style-based GANs on three medical imaging modalities and org…
As an effective way to integrate the information contained in multiple medical images under different modalities, medical image synthesis and fusion have emerged in various clinical applications such as disease diagnosis and treatment planning. In this paper, an invertible and variable augmented network (iVAN) is proposed for medical image synthesis and fusion. In iVAN, the channel number of the network input and output is the same through variable augmentation technology, and data relevance is enhanced, which is conducive to the generation of characterization information. Meanwhile, the inver…
College of Science and Engineering, Hamad Bin Khalifa University（哈马德·本·哈利法大学科学与工程学院）；Manchester University NHS Foundation Trust（曼彻斯特大学国民保健服务基金会信托）
Generative models are becoming popular for the synthesis of medical images. Recently, neural diffusion models have demonstrated the potential to generate photo-realistic images of objects. However, their potential to generate medical images is not explored yet. In this work, we explore the possibilities of synthesis of medical images using neural diffusion models. First, we use a pre-trained DALLE2 model to generate lungs X-Ray and CT images from an input text prompt. Second, we train a stable diffusion model with 3165 X-Ray images and generate synthetic images. We evaluate the synthetic image…
A Generalist Model for Diverse Text-Guided Medical Image Synthesis
Department of Cardiothoracic Surgery（心脏外科部门）；Stanford Medicine（斯坦福医学）；Department of Ophthalmology（眼科部门）；Division of Cardiovascular Surgery（心血管外科分会）；Penn Medicine（宾夕法尼亚医学）
Pseudo-normality synthesis, which computationally generates a pseudo-normal image from an abnormal one (e.g., with lesions), is critical in many perspectives, from lesion detection, data augmentation to clinical surgery suggestion. However, it is challenging to generate high-quality pseudo-normal images in the absence of the lesion information. Thus, expensive lesion segmentation data have been introduced to provide lesion information for the generative models and improve the quality of the synthetic images. In this paper, we aim to alleviate the need of a large amount of lesion segmentation d…
NVIDIA Corporation（英伟达公司）；MGH & BWH Center for Clinical Data Science（麻省总医院与布列根和妇女医院临床数据科学中心）；Mayo Clinic（妙佑医疗国际）
Data diversity is critical to success when training deep learning models. Medical imaging data sets are often imbalanced as pathologic findings are generally rare, which introduces significant challenges when training deep learning models. In this work, we propose a method to generate synthetic abnormal MRI images with brain tumors by training a generative adversarial network using two publicly available data sets of brain MRI. We demonstrate two unique benefits that the synthetic images provide. First, we illustrate improved performance on tumor segmentation by leveraging the synthetic images…
Maynooth University（梅努斯大学）；Dublin City University（都柏林城市大学）
Continuous medical time series data such as ECG is one of the most complex time series due to its dynamic and high dimensional characteristics. In addition, due to its sensitive nature, privacy concerns and legal restrictions, it is often even complex to use actual data for different medical research. As a result, generating continuous medical time series is a very critical research area. Several research works already showed that the ability of generative adversarial networks (GANs) in the case of continuous medical time series generation is promising. Most medical data generation works, such…
Universitat de Barcelona（巴塞罗那大学）；University of Arkansas for Medical Sciences（阿肯色医科大学）
Synthetic data generated by generative models can enhance the performance and capabilities of data-hungry deep learning models in medical imaging. However, there is (1) limited availability of (synthetic) datasets and (2) generative models are complex to train, which hinders their adoption in research and clinical applications. To reduce this entry barrier, we propose medigan, a one-stop shop for pretrained generative models implemented as an open-source framework-agnostic Python library. medigan allows researchers and developers to create, increase, and domain-adapt their training data in jus…
Unified Cross-Modal Medical Image Synthesis with Hierarchical Mixture of Product-of-Experts
Harvard Medical School and the Brigham and Women’s Hospital（哈佛医学院和布里洛妇产科医院）；MIND Team, Inria Saclay, Université Paris-Saclay, Palaiseau, France and Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, CNRS, Inria, Inserm, AP-HP, Hôpital de la Pitié Salpêtrière（MIND团队、Inria Saclay、巴黎萨克雷大学、Palaiseau，法国以及索邦大学、巴黎脑研究所-ICM、CNRS、Inria、Inserm、AP-HP、皮蒂埃萨勒普蒂耶医院）；Massachusetts Institute of Technology（麻省理工学院）
Medical image annotation typically requires expert knowledge and hence incurs time-consuming and expensive data annotation costs. To alleviate this burden, we propose a novel learning scenario, Exemplar Learning (EL), to explore automated learning processes for medical image segmentation with a single annotated image example. This innovative learning task is particularly suitable for medical image segmentation, where all categories of organs can be presented in one single image and annotated all at once. To address this challenging EL task, we propose an Exemplar Learning-based Synthesis Net (…
National Heart and Lung Institute, Imperial College London（帝国理工学院国家心肺研究所）；Royal Brompton Hospital（皇家布朗普顿医院）；Centre for Cardiovascular Science, University of Edinburgh（爱丁堡大学心血管科学中心）；School of Computer Science and Electronic Engineering, University of Essex（埃塞克斯大学计算机科学与电子工程学院）
As a pragmatic data augmentation tool, data synthesis has generally returned dividends in performance for deep learning based medical image analysis. However, generating corresponding segmentation masks for synthetic medical images is laborious and subjective. To obtain paired synthetic medical images and segmentations, conditional generative models that use segmentation masks as synthesis conditions were proposed. However, these segmentation mask-conditioned generative models still relied on large, varied, and labeled training datasets, and they could only provide limited constraints on human…
Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）
In this paper, we leverage predictive uncertainty of deep neural networks to answer challenging questions material scientists usually encounter in machine learning based materials applications workflows. First, we show that by leveraging predictive uncertainty, a user can determine the required training data set size necessary to achieve a certain classification accuracy. Next, we propose uncertainty guided decision referral to detect and refrain from making decisions on confusing samples. Finally, we show that predictive uncertainty can also be used to detect out-of-distribution test samples.…
LMU Munich（慕尼黑大学）；University Hospital, LMU Munich（慕尼黑大学附属医院）；Munich Center for Machine Learning（慕尼黑机器学习中心）
While recent advances in large-scale foundational models show promising results, their application to the medical domain has not yet been explored in detail. In this paper, we progress into the realms of large-scale modeling in medical synthesis by proposing Cheff - a foundational cascaded latent diffusion model, which generates highly-realistic chest radiographs providing state-of-the-art quality on a 1-megapixel scale. We further propose MaCheX, which is a unified interface for public chest datasets and forms the largest open collection of chest X-rays up to date. With Cheff conditioned on r…
Modern medicine requires generalised approaches to the synthesis and integration of multimodal data, often at different biological scales, that can be applied to a variety of evidence structures, such as complex disease analyses and epidemiological models. However, current methods are either slow and expensive, or ineffective due to the inability to model the complex relationships between data modes which differ in scale and format. We address these issues by proposing a cross-modal deep learning architecture and co-attention mechanism to accurately model the relationships between the differen…
