CLIP-Inspector：通过OoD触发器反向工程实现提示调优CLIP的模型级后门检测
CLIP-Inspector: Model-Level Backdoor Detection for Prompt-Tuned CLIP via OOD Trigger Inversion
AI总结：
针对提示调优CLIP模型中潜在的后门攻击，CLIP-Inspector通过OoD触发器反向工程实现模型级后门检测，利用白盒访问和未标记OoD图像重建可能触发器，验证模型安全性并修复后门效果。
摘要：
拥有有限数据和计算资源的组织越来越多地将模型训练外包给机器学习即服务（MLaaS）提供商，这些提供商通过提示调优而非从头训练来适应视觉语言模型（VLMs）如CLIP以执行下游任务。这种半诚实的设置会产生安全风险，恶意提供商可能遵循提示调优协议却植入后门，迫使触发输入被分类为攻击者选择的类别，即使对于分布外（OOD）数据。此类后门不会影响编码器，使现有专注于编码器损坏的方法无法检测。其他数据级方法在训练前或推理期间清洗数据也未能回答关键问题：“交付的模型是否被后门污染？”为了解决这个模型级验证问题，我们引入CLIP-Inspector（CI），一种针对提示调优CLIP模型的后门检测方法。假设对交付模型有白盒访问权限，并且有一个未标记的OOD图像池，CI重建每个类别的可能触发器以确定模型是否表现出后门行为。此外，我们展示了使用CI重建的触发器对正确标记的触发输入进行微调，可以重新对齐模型并减少后门的有效性。通过在十个数据集和四种后门攻击上的广泛实验，我们证明CI可以在仅使用1,000个OOD图像的情况下，在单个epoch内重建有效的触发器，达到94%的检测准确率（47/50个模型）。与适应触发器反向工程基线相比，CI的AUROC分数显著更高（0.973 vs 0.495/0.687），从而能够验证和事后修复提示调优CLIP模型以确保安全部署。
英文摘要：
Organisations with limited data and computational resources increasingly outsource model training to Machine Learning as a Service (MLaaS) providers, who adapt vision-language models (VLMs) such as CLIP to downstream tasks via prompt tuning rather than training from scratch. This semi-honest setting creates a security risk where a malicious provider can follow the prompt-tuning protocol yet implant a backdoor, forcing triggered inputs to be classified into an attacker-chosen class, even for out-of-distribution (OOD) data. Such backdoors leave encoders untouched, making them undetectable to existing methods that focus on encoder corruption. Other data-level methods that sanitize data before training or during inference, also fail to answer the critical question, "Is the delivered model backdoored or not?" To address this model-level verification problem, we introduce CLIP-Inspector (CI), a backdoor detection method designed for prompt-tuned CLIP models. Assuming white-box access to the delivered model and a pool of unlabeled OOD images, CI reconstructs possible triggers for each class to determine if the model exhibits backdoor behaviour or not. Additionally, we demonstrate that using CI's reconstructed trigger for fine-tuning on correctly labeled triggered inputs enables us to re-align the model and reduce backdoor effectiveness. Through extensive experiments across ten datasets and four backdoor attacks, we demonstrate that CI can reconstruct effective triggers in a single epoch using only 1,000 OOD images, achieving a 94% detection accuracy (47/50 models). Compared to adapted trigger-inversion baselines, CI yields a markedly higher AUROC score (0.973 vs 0.495/0.687), thus enabling the vetting and post-hoc repair of prompt-tuned CLIP models to ensure safe deployment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
