受神经启发的多模态视觉-语言模型对成员推断隐私泄露是否具有弹性？
Are Neuro-Inspired Multi-Modal Vision-Language Models Resilient to Membership Inference Privacy Leakage?
AI总结：
研究受神经启发的多模态视觉-语言模型（VLM）对基于图像-文本的成员推断攻击的弹性，提出拓扑正则化框架，实验表明神经VLM在保持模型效用同时显著降低攻击成功率。
摘要：
在智能体AI时代，多模态模型（MMs）的日益部署引入了新的攻击向量，可能导致MMs中敏感训练数据泄露，造成隐私泄露。本文研究了一种黑盒隐私攻击，即对多模态视觉-语言模型（VLMs）的成员推断攻击（MIA）。最先进的研究主要分析单模态AI-ML系统的隐私攻击，而最近的研究表明MMs也可能易受隐私攻击。尽管研究人员已证明生物启发的神经网络表示可以提高单模态模型对对抗攻击的弹性，但受神经启发的MMs是否对隐私攻击具有弹性仍未被探索。在这项工作中，我们引入了一个系统的神经科学启发的拓扑正则化（τ）框架，以分析MM VLMs对基于图像-文本的推断隐私攻击的弹性。我们使用三个VLM：BLIP、PaliGemma 2和ViT-GPT2，在三个基准数据集：COCO、CC3M和NoCaps上检验了这一现象。我们的实验比较了基线VLM和神经VLM（带有拓扑正则化）的弹性，其中τ>0配置定义了VLM的NEURO变体。我们在COCO数据集上使用BLIP模型的结果表明，NEURO VLM中MIA攻击成功率平均下降24%的ROC-AUC，同时在MPNet和ROUGE-2指标上实现了相似的模型效用（生成字幕与参考字幕之间的相似性）。这表明神经VLM相对更具隐私攻击弹性，同时不会显著牺牲模型效用。我们使用PaliGemma 2和ViT-GPT2模型在另外两个数据集CC3M和NoCaps上的广泛评估进一步验证了发现的一致性。这项工作有助于加深对MMs中隐私风险的理解，并为神经VLM的隐私威胁弹性提供了证据。
英文摘要：
In the age of agentic AI, the growing deployment of multi-modal models (MMs) has introduced new attack vectors that can leak sensitive training data in MMs, causing privacy leakage. This paper investigates a black-box privacy attack, i.e., membership inference attack (MIA) on multi-modal vision-language models (VLMs). State-of-the-art research analyzes privacy attacks primarily to unimodal AI-ML systems, while recent studies indicate MMs can also be vulnerable to privacy attacks. While researchers have demonstrated that biologically inspired neural network representations can improve unimodal model resilience against adversarial attacks, it remains unexplored whether neuro-inspired MMs are resilient against privacy attacks. In this work, we introduce a systematic neuroscience-inspired topological regularization (tau) framework to analyze MM VLMs resilience against image-text-based inference privacy attacks. We examine this phenomenon using three VLMs: BLIP, PaliGemma 2, and ViT-GPT2, across three benchmark datasets: COCO, CC3M, and NoCaps. Our experiments compare the resilience of baseline and neuro VLMs (with topological regularization), where the tau > 0 configuration defines the NEURO variant of VLM. Our results on the BLIP model using the COCO dataset illustrate that MIA attack success in NEURO VLMs drops by 24% mean ROC-AUC, while achieving similar model utility (similarities between generated and reference captions) in terms of MPNet and ROUGE-2 metrics. This shows neuro VLMs are comparatively more resilient against privacy attacks, while not significantly compromising model utility. Our extensive evaluation with PaliGemma 2 and ViT-GPT2 models, on two additional datasets: CC3M and NoCaps, further validates the consistency of the findings. This work contributes to the growing understanding of privacy risks in MMs and provides evidence on neuro VLMs privacy threat resilience.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
