医学图像分割中视觉-语言模型的对抗鲁棒性分析
Adversarial Robustness Analysis of Vision-Language Models in Medical Image Segmentation
AI总结：
本文研究医学图像分割中视觉-语言模型对PGD和FGSM对抗攻击的鲁棒性，通过适配器微调VLSMs并报告DSC和IoU显著下降，揭示其在高风险医学场景中的脆弱性。
摘要：
对抗攻击在计算机视觉和视觉-语言模型中已得到较多研究，但针对视觉-语言分割模型（VLSMs）的对抗攻击途径仍探索不足，尤其是在医学图像分析领域。因此，我们研究了VLSMs在二维医学图像上面对对抗攻击的鲁棒性，图像模态涵盖放射学、摄影和内窥镜。本项目的主要思路是评估在医学领域设置下经过微调的VLSMs的鲁棒性，以应对高风险场景。首先，我们使用适配器对预训练的VLSMs进行微调，用于医学图像分割。然后，我们在微调后的模型上施加对抗攻击——投影梯度下降（PGD）和快速梯度符号法（FGSM）——以确定其对对抗样本的鲁棒性。我们报告了模型性能的下降情况，以分析对抗样本的影响。结果表明，在引入这些对抗样本后，DSC和IoU分数出现显著下降。此外，我们还探索了通用扰动，但未能在医学图像中找到有效的通用扰动。
英文摘要：
Adversarial attacks have been fairly explored for computer vision and vision-language models. However, the avenue of adversarial attack for the vision language segmentation models (VLSMs) is still under-explored, especially for medical image analysis. Thus, we have investigated the robustness of VLSMs against adversarial attacks for 2D medical images with different modalities with radiology, photography, and endoscopy. The main idea of this project was to assess the robustness of the fine-tuned VLSMs specially in the medical domain setting to address the high risk scenario. First, we have fine-tuned pre-trained VLSMs for medical image segmentation with adapters. Then, we have employed adversarial attacks -- projected gradient descent (PGD) and fast gradient sign method (FGSM) -- on that fine-tuned model to determine its robustness against adversaries. We have reported models' performance decline to analyze the adversaries' impact. The results exhibit significant drops in the DSC and IoU scores after the introduction of these adversaries. Furthermore, we also explored universal perturbation but were not able to find for the medical images. \footnote{https://github.com/anjilab/secure-private-ai}
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
