The shift towards efficient and automated data analysis through Machine Learning (ML) has notably impacted healthcare systems, particularly Radiomics. Radiomics leverages ML to analyze medical images accurately and efficiently for precision medicine. Current methods rely on Deep Learning (DL) to improve performance and accuracy (Deep Radiomics). Given the sensitivity of medical images, ensuring privacy throughout the Deep Radiomics pipeline-from data generation and collection to model training and inference-is essential, especially when outsourced. Thus, Privacy-Enhancing Technologies (PETs) are crucial tools for Deep Radiomics. Previous studies and systematization efforts have either broadly overviewed PETs and their applications or mainly focused on subsets of PETs for ML algorithms. In Deep Radiomics, where efficiency, accuracy, and privacy are crucial, many PETs, while theoretically applicable, may not be practical without specialized optimizations or hybrid designs. Additionally, not all DL models are suitable for Radiomics. Consequently, there is a need for specialized studies that investigate and systematize the effective and practical integration of PETs into the Deep Radiomics pipeline. This work addresses this research gap by (1) classifying existing PETs, presenting practical hybrid PETS constructions, and a taxonomy illustrating their potential integration with the Deep Radiomics pipeline, with comparative analyses detailing assumptions, architectural suitability, and security, (2) Offering technical insights, describing potential challenges and means of combining PETs into the Deep Radiomics pipeline, including integration strategies, subtilities, and potential challenges, (3) Proposing potential research directions, identifying challenges, and suggesting solutions to enhance the PETs in Deep Radiomics.
University of Waterloo（滑铁卢大学）；National Research Council Canada（加拿大国家研究委员会）
Computer vision and machine learning are playing an increasingly important role in computer-assisted diagnosis; however, the application of deep learning to medical imaging has challenges in data availability and data imbalance, and it is especially important that models for medical imaging are built to be trustworthy. Therefore, we propose TRUDLMIA, a trustworthy deep learning framework for medical image analysis, which adopts a modular design, leverages self-supervised pre-training, and utilizes a novel surrogate loss function. Experimental evaluations indicate that models generated from the…
Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；Inserm（法国国家健康与医学研究院）；Pixyl Research and Development Laboratory（Pixyl研发实验室）；Inria（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔理工学院）
The full acceptance of Deep Learning (DL) models in the clinical field is rather low with respect to the quantity of high-performing solutions reported in the literature. Particularly, end users are reluctant to rely on the rough predictions of DL models. Uncertainty quantification methods have been proposed in the literature as a potential response to reduce the rough decision provided by the DL black box and thus increase the interpretability and the acceptability of the result by the final user. In this review, we propose an overview of the existing methods to quantify uncertainty associate…
East China Normal University（华东师范大学）；The University of Adelaide（阿德莱德大学）；The University of New South Wales（新南威尔士大学）；Data61-CSIRO（澳大利亚联邦科学与工业研究组织数据61中心）
Deep Neural Networks have achieved unprecedented success in the field of face recognition such that any individual can crawl the data of others from the Internet without their explicit permission for the purpose of training high-precision face recognition models, creating a serious violation of privacy. Recently, a well-known system named Fawkes (published in USENIX Security 2020) claimed this privacy threat can be neutralized by uploading cloaked user images instead of their original images. In this paper, we present Oriole, a system that combines the advantages of data poisoning attacks and …
Federated Distillation for Medical Image Classification: Towards Trustworthy Computer-Aided Diagnosis
School of Information and Communication Engineering, Hainan University（海南大学信息与通信工程学院）；College of Mathematical Sciences, BoHai University（渤海大学数学科学学院）；Australian AI Institute, University of Technology Sydney（悉尼科技大学澳大利亚人工智能研究所）；School of Electronic Science and Technology, Hainan University（海南大学电子科学与技术学院）
FUTURE-AI: Guiding Principles and Consensus Recommendations for Trustworthy Artificial Intelligence in Medical Imaging
Barcelona Artificial Intelligence in Medicine Lab (BCN-AIM)（巴塞罗那医学人工智能实验室（BCN-AIM））；Maggioli SPA（马焦利股份公司）；Fundació TIC Salut i Social（信息与通信技术社会基金会）；Generalitat de Catalunya（加泰罗尼亚政府）；Hospital Universitario y Politécnico La Fe（拉费大学及综合医院）；Health Research Institute（健康研究所）；Foundation for Research and Technology（研究与技术基金会）；National Research Council（国家研究委员会）；Maastricht University（马斯特里赫特大学）
Johns Hopkins University（约翰霍普金斯大学）；Applied Physics Laboratory（应用物理实验室）；University of Virginia（弗吉尼亚大学）
Machine learning (ML) models used in medical imaging diagnostics can be vulnerable to a variety of privacy attacks, including membership inference attacks, that lead to violations of regulations governing the use of medical data and threaten to compromise their effective deployment in the clinic. In contrast to most recent work in privacy-aware ML that has been focused on model alteration and post-processing steps, we propose here a novel and complementary scheme that enhances the security of medical data by controlling the data sharing process. We develop and evaluate a privacy defense protoc…
University of Waterloo（滑铁卢大学）；National Research Council Canada（加拿大国家研究委员会）
Building AI models with trustworthiness is important especially in regulated areas such as healthcare. In tackling COVID-19, previous work uses convolutional neural networks as the backbone architecture, which has shown to be prone to over-caution and overconfidence in making decisions, rendering them less trustworthy -- a crucial flaw in the context of medical imaging. In this study, we propose a feature learning approach using Vision Transformers, which use an attention-based mechanism, and examine the representation learning capability of Transformers as a new backbone architecture for medi…
OpenMined；Manifold Computing Group（曼尼福德计算集团）；Ford R&A（福特研发部门）；Harvard University（哈佛大学）；Indian Institute of Technology Jodhpur（印度理工学院焦特布尔分校）；University of Oxford（牛津大学）
In this paper we measure the effectiveness of $ε$-Differential Privacy (DP) when applied to medical imaging. We compare two robust differential privacy mechanisms: Local-DP and DP-SGD and benchmark their performance when analyzing medical imagery records. We analyze the trade-off between the model's accuracy and the level of privacy it guarantees, and also take a closer look to evaluate how useful these theoretical privacy guarantees actually prove to be in the real world medical setting.
University of Porto（波尔图大学）；INESC TEC（葡萄牙系统与计算机工程研究所）
The use of Deep Learning in the medical field is hindered by the lack of interpretability. Case-based interpretability strategies can provide intuitive explanations for deep learning models' decisions, thus, enhancing trust. However, the resulting explanations threaten patient privacy, motivating the development of privacy-preserving methods compatible with the specifics of medical data. In this work, we analyze existing privacy-preserving methods and their respective capacity to anonymize medical data while preserving disease-related semantic features. We find that the PPRL-VGAN deep learning…
École de Technologie Supérieure（高等技术学院）；Université de Sherbrooke（舍布鲁克大学）
This paper presents a client/server privacy-preserving network in the context of multicentric medical image analysis. Our approach is based on adversarial learning which encodes images to obfuscate the patient identity while preserving enough information for a target task. Our novel architecture is composed of three components: 1) an encoder network which removes identity-specific features from input medical images, 2) a discriminator network that attempts to identify the subject from the encoded images, 3) a medical image analysis network which analyzes the content of the encoded images (segm…
A Dempster-Shafer approach to trustworthy AI with application to fetal brain MRI segmentation
University Hospitals Leuven（鲁汶大学医院）；Technical University Munich（慕尼黑工业大学）；University Hospital Zurich（苏黎世大学医院）；University of Zurich（苏黎世大学）；University College London（伦敦大学学院）；University Hospitals Tanta（坦塔大学医院）；Medical University of Vienna（维也纳医科大学）
Multi-Modal Explainable Medical AI Assistant for Trustworthy Human-AI Collaboration
Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology（电子与计算机工程系，香港科学与技术大学）；Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（计算机科学与工程系，香港科学与技术大学）
Technical University of Munich（慕尼黑工业大学）；Imperial College London（帝国理工学院）；Arkhn；INRIA（法国国家信息与自动化研究所）；ENS（巴黎高等师范学院）；PSL University（巴黎文理研究大学）；University of Oxford（牛津大学）；Universidade Federal de Campina Grande（坎皮纳格兰德联邦大学）；Cape Privacy
The utilisation of artificial intelligence in medicine and healthcare has led to successful clinical applications in several domains. The conflict between data usage and privacy protection requirements in such systems must be resolved for optimal results as well as ethical and legal compliance. This calls for innovative solutions such as privacy-preserving machine learning (PPML). We present PriMIA (Privacy-preserving Medical Image Analysis), a software framework designed for PPML in medical imaging. In a real-life case study we demonstrate significantly better classification performance of a …
Differential privacy representation geometry for medical image analysis
Lab for AI in Medicine, RWTH Aachen University, Aachen, Germany（医学人工智能实验室，亚琛RWTH大学，亚琛，德国）；Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany（诊断与介入放射学部门，RWTH亚琛大学医院，亚琛，德国）；INDA Institute, RWTH Aachen University, Aachen, Germany（INDA研究所，亚琛RWTH大学，亚琛，德国）
Differential privacy for medical deep learning: methods, tradeoffs, and deployment implications
Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg（埃朗根-纽伦堡弗里德里希-亚历山大大学模式识别实验室）；Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen（亚琛大学诊断与介入放射科）；Department of Radiology, Stanford University（斯坦福大学放射科）；Department of Urology, Stanford University（斯坦福大学泌尿科）
The privacy risks of machine learning models is a major concern when training them on sensitive and personal data. We discuss the tradeoffs between data privacy and the remaining goals of trustworthy machine learning (notably, fairness, robustness, and explainability).
Software Competence Center Hagenberg（哈根贝格软件能力中心）；University of Rostock（罗斯托克大学）
In order to develop machine learning and deep learning models that take into account the guidelines and principles of trustworthy AI, a novel information theoretic trustworthy AI framework is introduced. A unified approach to "privacy-preserving interpretable and transferable learning" is considered for studying and optimizing the tradeoffs between privacy, interpretability, and transferability aspects. A variational membership-mapping Bayesian model is used for the analytical approximations of the defined information theoretic measures for privacy-leakage, interpretability, and transferabilit…
Decentralised cloud computing platforms enable individuals to offer and rent resources in a peer-to-peer fashion. They must assign resources from multiple sellers to multiple buyers and derive prices that match the interests and capacities of both parties. The assignment process must be decentralised, fair and transparent, but also protect the privacy of buyers. We present PASTRAMI, a decentralised platform enabling trustworthy assignments of items and prices between a large number of sellers and bidders, through the support of multi-item auctions. PASTRAMI uses threshold blind signatures and …
The Chinese University of Hong Kong（香港中文大学）；University of Electronic Science and Technology of China（电子科技大学）；Peng Cheng Lab（鹏城实验室）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））
Trustworthy artificial intelligence (AI) technology has revolutionized daily life and greatly benefited human society. Among various AI technologies, Federated Learning (FL) stands out as a promising solution for diverse real-world scenarios, ranging from risk evaluation systems in finance to cutting-edge technologies like drug discovery in life sciences. However, challenges around data isolation and privacy threaten the trustworthiness of FL systems. Adversarial attacks against data privacy, learning algorithm stability, and system confidentiality are particularly concerning in the context of…
In this paper, we introduce an efficient and trustworthy conditional privacy-preserving communication protocol for VANETs based on proxy re-signature. The proposed protocol is characterized by the Trusted Authority (TA) designating the Roadside Units (RSUs) to translate signatures computed by the On-Board Units (OBUs) into one that are valid with respect to TA's public key. In addition, the proposed protocol offers both a priori and a posteriori countermeasures: it can not only provide fast anonymous authentication and privacy tracking, but guarantees message trustworthiness for vehicle-to-veh…
Many successful services rely on trustworthy contributions from users. To establish that trust, such services often require access to privacy-sensitive information from users, thus creating a conflict between privacy and trust. Although it is likely impractical to expect both absolute privacy and trustworthiness at the same time, we argue that the current state of things, where individual privacy is usually sacrificed at the altar of trustworthy services, can be improved with a pragmatic $Glimmer$ $of$ $Trust$, which allows services to validate user contributions in a trustworthy way without f…
The Internet of Things (IoT) is considered as the key enabling technology for smart services. Security and privacy are particularly open challenges for IoT applications due to the widespread use of commodity devices. This work introduces two hardware-based lightweight security mechanisms to ensure sensed data trustworthiness (i.e., sensed data protection and sensor node protection) and usage privacy of the sensors (i.e., privacy-aware reporting of the sensed data) for centralized and decentralized IoT applications. Physically unclonable functions (PUF) form the basis of both proposed mechanism…
Trustworthy Federated Learning: Privacy, Security, and Beyond
HiThink Research（嗨思研究院）；Carnegie Mellon University（卡内基梅隆大学）；Kyushu University（九州大学）；The University of Auckland（奥克兰大学）；The University of Aizu（会津大学）；Zhejiang Sci-Tech University（浙江理工大学）；Fudan University（复旦大学）；BEDI Cloud（贝迪云）
Institute for Artificial Intelligence in Medicine, University Hospital Essen（埃森大学医院医学人工智能研究所）；University of Duisburg-Essen（杜伊斯堡-埃森大学）；Cancer Research Center Cologne Essen（科隆-埃森癌症研究中心）
AI/Computing at scale is a difficult problem, especially in a health care setting. We outline the requirements, planning and implementation choices as well as the guiding principles that led to the implementation of our secure research computing enclave, the Essen Medical Computing Platform (EMCP), affiliated with a major German hospital. Compliance, data privacy and usability were the immutable requirements of the system. We will discuss the features of our computing enclave and we will provide our recipe for groups wishing to adopt a similar setup.
Towards trustworthy agentic AI: a comprehensive survey of safety, robustness, privacy, and system security
Faculty of Engineering, Department of Computer Science and Engineering, The Chinese University of Hong Kong（香港中文大学工程学院、计算机科学与工程系）；Artificial Intelligence Innovation and Incubation Institute, Fudan University（复旦大学人工智能创新与孵化院）；Shanghai Academy of AI for Science（上海人工智能科学研究院）
Security and Privacy in Retrieval-Augmented Generation: Architectures, Threats, Defenses, and Future Directions for Building Trustworthy Systems
Department of Electrical and Electronics Engineering（电子与电气工程系）；Birla Institute of Technology and Science, Pilani, Pilani Campus（比拉理工学院和科学学院，比拉校区）；Department of Computer Engineering, KIIT University（计算机工程系，KIIT大学）；Quantum Cloud Computing and Distributed Systems (qCLOUDS) Laboratory（量子云计算与分布式系统（qCLOUDS）实验室）；Department of Computing and Information Systems（计算与信息系统系）；The University of Melbourne（墨尔本大学）
Decentralized systems can be more resistant to operator mischief than centralized ones, but they are substantially harder to develop, deploy, and maintain. This cost is dramatically reduced if the decentralized part of the system can be made highly generic, and thus incorporated into many different applications. We show how existing anonymization systems can serve this purpose, securing a public database against equivocation by its operator without the need for cooperation by the database owner. We derive bounds on the probability of successful equivocation, and in doing so, we demonstrate tha…
