Adversarial Attack Vulnerability of Medical Image Analysis Systems: Unexplored Factors
Erasmus MC（伊拉斯谟医学中心）；Radboudumc（拉德堡德大学医学中心）；Donders Institute for Brain, Cognition and Behaviour（唐德斯大脑、认知与行为研究所）；Eindhoven University of Technology（埃因霍温理工大学）；Intel Corporation（英特尔公司）
Gerda Bortsova, Cristina González-Gonzalo, Suzanne C. Wetstein, Florian Dubost, Ioannis Katramados, Laurens Hogeweg, Bart Liefers, Bram van Ginneken, Josien P. … 展开作者
Gerda Bortsova, Cristina González-Gonzalo, Suzanne C. Wetstein, Florian Dubost, Ioannis Katramados, Laurens Hogeweg, Bart Liefers, Bram van Ginneken, Josien P. W. Pluim, Mitko Veta, Clara I. Sánchez, Marleen de Bruijne
备注：
First three authors contributed equally
发表信息：
Medical Image Analysis. Available online 18 Jun 2021
Adversarial attacks are considered a potentially serious security threat for machine learning systems. Medical image analysis (MedIA) systems have recently been argued to be vulnerable to adversarial attacks due to strong financial incentives and the associated technological infrastructure.
In this paper, we study previously unexplored factors affecting adversarial attack vulnerability of deep learning MedIA systems in three medical domains: ophthalmology, radiology, and pathology. We focus on adversarial black-box settings, in which the attacker does not have full access to the target model and usually uses another model, commonly referred to as surrogate model, to craft adversarial examples. We consider this to be the most realistic scenario for MedIA systems.
Firstly, we study the effect of weight initialization (ImageNet vs. random) on the transferability of adversarial attacks from the surrogate model to the target model. Secondly, we study the influence of differences in development data between target and surrogate models. We further study the interaction of weight initialization and data differences with differences in model architecture. All experiments were done with a perturbation degree tuned to ensure maximal transferability at minimal visual perceptibility of the attacks.
Our experiments show that pre-training may dramatically increase the transferability of adversarial examples, even when the target and surrogate's architectures are different: the larger the performance gain using pre-training, the larger the transferability. Differences in the development data between target and surrogate models considerably decrease the performance of the attack; this decrease is further amplified by difference in the model architecture. We believe these factors should be considered when developing security-critical MedIA systems planned to be deployed in clinical practice.
Mohamed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Vision Transformers (ViT) are competing to replace Convolutional Neural Networks (CNN) for various computer vision tasks in medical imaging such as classification and segmentation. While the vulnerability of CNNs to adversarial attacks is a well-known problem, recent works have shown that ViTs are also susceptible to such attacks and suffer significant performance degradation under attack. The vulnerability of ViTs to carefully engineered adversarial samples raises serious concerns about their safety in clinical settings. In this paper, we propose a novel self-ensembling method to enhance the …
Adversarial Medical Image with Hierarchical Feature Hiding
University of Science and Technology of China（中国科学技术大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Tencent YouTu Lab（腾讯优图实验室）；Guangxi Medical University（广西医科大学）；The Hong Kong University of Science and Technology（香港科技大学）；Meta Reality Labs（Meta现实实验室）
ETH Zürich（苏黎世联邦理工学院）；University of Cambridge（剑桥大学）
Semantic segmentation of medical images is a crucial step for the quantification of healthy anatomy and diseases alike. The majority of the current state-of-the-art segmentation algorithms are based on deep neural networks and rely on large datasets with full pixel-wise annotations. Producing such annotations can often only be done by medical professionals and requires large amounts of valuable time. Training a medical image segmentation network with weak annotations remains a relatively unexplored topic. In this work we investigate training strategies to learn the parameters of a pixel-wise s…
Arizona State University（亚利桑那州立大学）；Mayo Clinic（梅奥诊所）
Medical images are naturally associated with rich semantics about the human anatomy, reflected in an abundance of recurring anatomical patterns, offering unique potential to foster deep semantic representation learning and yield semantically more powerful models for different medical applications. But how exactly such strong yet free semantics embedded in medical images can be harnessed for self-supervised learning remains largely unexplored. To this end, we train deep models to learn semantically enriched visual representation by self-discovery, self-classification, and self-restoration of th…
University of Sydney（悉尼大学）；Shanghai Jiao Tong University（上海交通大学）；UNSW（新南威尔士大学）
Gigapixel medical images provide massive data, both morphological textures and spatial information, to be mined. Due to the large data scale in histology, deep learning methods play an increasingly significant role as feature extractors. Existing solutions heavily rely on convolutional neural networks (CNNs) for global pixel-level analysis, leaving the underlying local geometric structure such as the interaction between cells in the tumor microenvironment unexplored. The topological structure in medical images, as proven to be closely related to tumor evolution, can be well characterized by gr…
Universal adversarial attacks, which hinder most deep neural network (DNN) tasks using only a small single perturbation called a universal adversarial perturbation (UAP), is a realistic security threat to the practical application of a DNN. In particular, such attacks cause serious problems in medical imaging. Given that computer-based systems are generally operated under a black-box condition in which only queries on inputs are allowed and outputs are accessible, the impact of UAPs seems to be limited because well-used algorithms for generating UAPs are limited to a white-box condition in whi…
BRAC University（BRAC大学）；Islamic University of Technology（伊斯兰技术大学）
In the world of medical diagnostics, the adoption of various deep learning techniques is quite common as well as effective, and its statement is equally true when it comes to implementing it into the retina Optical Coherence Tomography (OCT) sector, but (i)These techniques have the black box characteristics that prevent the medical professionals to completely trust the results generated from them (ii)Lack of precision of these methods restricts their implementation in clinical and complex cases (iii)The existing works and models on the OCT classification are substantially large and complicated…
The Chinese University of Hong Kong（香港中文大学）；Tencent Jarvis Lab（腾讯医疗健康（深圳）有限公司（腾讯天衍实验室））；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；The University of Hong Kong（香港大学）；Tsinghua University（清华大学）；Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）
Semi-supervised learning has substantially advanced medical image segmentation since it alleviates the heavy burden of acquiring the costly expert-examined annotations. Especially, the consistency-based approaches have attracted more attention for their superior performance, wherein the real labels are only utilized to supervise their paired images via supervised loss while the unlabeled images are exploited by enforcing the perturbation-based \textit{"unsupervised"} consistency without explicit guidance from those real labels. However, intuitively, the expert-examined real labels contain more…
FairDomain: Achieving Fairness in Cross-Domain Medical Image Segmentation and Classification
Center for Artificial Intelligence and Robotics, New York University Abu Dhabi（纽约大学阿布扎比分校人工智能与机器人中心）；New York University（纽约大学）；Harvard University（哈佛大学）
Sun Yat-sen University（中山大学）；Shuguang Hospital, Shanghai University of Traditional Chinese Medicine（上海中医药大学附属曙光医院）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
As the COVID-19 pandemic puts pressure on healthcare systems worldwide, the computed tomography image based AI diagnostic system has become a sustainable solution for early diagnosis. However, the model-wise vulnerability under adversarial perturbation hinders its deployment in practical situation. The existing adversarial training strategies are difficult to generalized into medical imaging field challenged by complex medical texture features. To overcome this challenge, we propose a Contour Attention Preserving (CAP) method based on lung cavity edge extraction. The contour prior features are…
TopoAgent: An Agentic Framework for Automated Topology Learning in Medical Imaging
Dept. of Computer Science and Engineering, University of Notre Dame（内布拉斯加大学达灵顿分校计算机科学与工程系）；Dept. of Computer Science, The University of Texas Rio Grande Valley（德克萨斯大学里奥格兰德谷分校计算机科学系）
University of California, Irvine（加州大学欧文分校）；Lenovo AI Lab（联想人工智能实验室）
Recently deep learning has been witnessing widespread adoption in various medical image applications. However, training complex deep neural nets requires large-scale datasets labeled with ground truth, which are often unavailable in many medical image domains. For instance, to train a deep neural net to detect pulmonary nodules in lung computed tomography (CT) images, current practice is to manually label nodule locations and sizes in many CT images to construct a sufficiently large training dataset, which is costly and difficult to scale. On the other hand, electronic medical records (EMR) co…
In this work, we defined an attack vector for networks utilizing the Internet of Medical Things (IoMT) devices and compute the probability distribution of IoMT security threats based on Markov chain and Common Vulnerability Scoring System (CVSS). IoMT is an emerging technology that improves patients' quality of life by permitting personalized e-health services without restrictions on time and site. The IoMT consists of embedded objects, sensors, and actuators that transmit and receive medical data. These Medical devices are vulnerable to different types of security threats, and thus, they pose…
Text Over Image: Auditing Multimodal Robustness in Synthetic Medical Image Detection
University of Notre Dame（圣母大学）；IBM Research（IBM研究院）；Boston Children’s Hospital（波士顿儿童医院）；Harvard Medical School（哈佛医学院）；Optum AI, UnitedHealth Group（Optum AI, 联合健康集团）
Advancing human-centric AI for robust X-ray analysis through holistic self-supervised learning
FAIR at Meta（Meta FAIR研究院）；Hôpital Cochin, AP-HP.Centre Université Paris Cité（科钦医院、巴黎大学中心公共医疗援助系统）；Université Paris Cité（巴黎大学）；CentraleSupélec, Université Paris-Saclay（巴黎萨克雷大学中央理工学院）
VEXAIoT: Autonomous IoT Vulnerability EXploitation using AI Agents
Department of Computer Science Tennessee Tech University Cookeville, TN, USA；School of Interdisciplinary Informatics University of Nebraska Omaha Omaha, NE, USA；1 Department of Computer Science, Tennessee Tech University, TN, USA.；2 School of Interdisciplinary Informatics, University of Nebraska Omaha, NE, USA.；3 Dept. of Mathematics \& Computer Science, University of North Carolina at Pembroke, NC, USA.
Distribution Service (DDS) is a realtime peer-to-peer protocol that serves as a scalable middleware between distributed networked systems found in many Industrial IoT domains such as automotive, medical, energy, and defense. Since the initial ratification of the standard, specifications have introduced a Security Model and Service Plugin Interface (SPI) architecture, facilitating authenticated encryption and data centric access control while preserving interoperable data exchange. However, as Secure DDS v1.1, the default plugin specifications presently exchanges digitally signed capability lis…
University of California, Davis（加州大学戴维斯分校）；Visa Research（Visa研究院）；Linköping University（林雪平大学）
Adversarial attacks on a convolutional neural network (CNN) -- injecting human-imperceptible perturbations into an input image -- could fool a high-performance CNN into making incorrect predictions. The success of adversarial attacks raises serious concerns about the robustness of CNNs, and prevents them from being used in safety-critical applications, such as medical diagnosis and autonomous driving. Our work introduces a visual analytics approach to understanding adversarial attacks by answering two questions: (1) which neurons are more vulnerable to attacks and (2) which image features do t…
Ghent University（根特大学）；Ghent University Global Campus（根特大学全球校区）
Deep learning models, which are increasingly being used in the field of medical image analysis, come with a major security risk, namely, their vulnerability to adversarial examples. Adversarial examples are carefully crafted samples that force machine learning models to make mistakes during testing time. These malicious samples have been shown to be highly effective in misguiding classification tasks. However, research on the influence of adversarial examples on segmentation is significantly lacking. Given that a large portion of medical imaging problems are effectively segmentation problems, …
Harvard Medical School（哈佛医学院）；MIT（麻省理工学院）；Massachusetts Institute of Technology（麻省理工学院）
The discovery of adversarial examples has raised concerns about the practical deployment of deep learning systems. In this paper, we demonstrate that adversarial examples are capable of manipulating deep learning systems across three clinical domains. For each of our representative medical deep learning classifiers, both white and black box attacks were highly successful. Our models are representative of the current state of the art in medical computer vision and, in some cases, directly reflect architectures already seeing deployment in real world clinical settings. In addition to the technic…
CISPA Helmholtz Center for Information Security（亥姆霍兹信息安全中心）；Salesforce Research（赛富比研究）
Self-supervised representation learning techniques have been developing rapidly to make full use of unlabeled images. They encode images into rich features that are oblivious to downstream tasks. Behind their revolutionary representation power, the requirements for dedicated model designs and a massive amount of computation resources expose image encoders to the risks of potential model stealing attacks - a cheap way to mimic the well-trained encoder performance while circumventing the demanding requirements. Yet conventional attacks only target supervised classifiers given their predicted lab…
The digitization of medical records ushered in a new era of big data to clinical science, and with it the possibility that data could be shared, to multiply insights beyond what investigators could abstract from paper records. The need to share individual-level medical data to accelerate innovation in precision medicine continues to grow, and has never been more urgent, as scientists grapple with the COVID-19 pandemic. However, enthusiasm for the use of big data has been tempered by a fully appropriate concern for patient autonomy and privacy. That is, the ability to extract private or confide…
Federated learning has become prevalent in medical diagnosis due to its effectiveness in training a federated model among multiple health institutions (i.e. Data Islands (DIs)). However, increasingly massive DI-level poisoning attacks have shed light on a vulnerability in federated learning, which inject poisoned data into certain DIs to corrupt the availability of the federated model. Previous works on federated learning have been inadequate in ensuring the privacy of DIs and the availability of the final federated model. In this paper, we design a secure federated learning mechanism with mul…
Fine-Tuning Foundation Models with Federated Learning for Privacy Preserving Medical Time Series Forecasting
University of Central Florida（中佛罗里达大学）；KnowledgeVis, LLC（诺维吉斯有限责任公司）；University of Michigan（密歇根大学）；University of California, Davis（加州大学戴维斯分校）；U.S. Army Institute of Surgical Research（美国陆军外科研究所）；University of Central Florida College of Medicine（中佛罗里达大学医学院）
Prompt Injection Attacks on Large Language Models in Oncology
Else Kroener Fresenius Center for Digital Health, Technical University Dresden（德累斯顿工业大学Else Kroener Fresenius数字健康中心）；University Hospital RWTH Aachen（亚琛工业大学附属医院）；National Center for Tumor Diseases (NCT), Heidelberg University Hospital（海德堡大学附属医院国家肿瘤疾病中心）；Heidelberg University（海德堡大学）；Medical Faculty Mannheim, Heidelberg University（海德堡大学曼海姆医学院）；German Cancer Research Center（德国癌症研究中心）；University Medical Center Mainz（美因茨大学医学中心）；University Hospital Aachen（亚琛大学附属医院）；University Hospital Dresden（德累斯顿大学附属医院）
