BackdoorIDS: 预训练视觉编码器的零样本后门检测
BackdoorIDS: Zero-shot Backdoor Detection for Pretrained Vision Encoder
AI总结：
BackdoorIDS通过分析输入在逐步遮蔽过程中的嵌入序列变化，利用密度聚类检测后门攻击，实现对预训练视觉编码器的零样本后门检测，优于现有防御方法。
摘要：
自监督和多模态视觉编码器学习了强大的视觉表示，广泛应用于下游视觉任务和大视觉-语言模型。然而，下游用户常依赖第三方预训练编码器，其来源不确定，易受后门攻击。本文提出BackdoorIDS，一种简单有效的零样本后门检测方法。受注意力劫持与恢复启发，逐步遮蔽输入时，后门图像初始关注恶意触发特征，当遮蔽比例超过触发器鲁棒性阈值后，触发器被禁用，注意力迅速转向良性内容。此转变导致图像嵌入显著变化，而清洁图像嵌入在遮蔽过程中变化更平滑。BackdoorIDS通过提取遮蔽轨迹上的嵌入序列并应用密度聚类如DBSCAN来操作此信号。若输入的嵌入序列形成超过一个聚类，则标记为后门攻击。大量实验表明，BackdoorIDS在各种攻击类型、数据集和模型家族中均优于现有防御方法。值得注意的是，它是一种即插即用的方法，无需重新训练，在推理时完全零样本运行，兼容多种编码器架构，包括CNN、ViT、CLIP和LLaVA-1.5。
英文摘要：
Self-supervised and multimodal vision encoders learn strong visual representations that are widely adopted in downstream vision tasks and large vision-language models (LVLMs). However, downstream users often rely on third-party pretrained encoders with uncertain provenance, exposing them to backdoor attacks. In this work, we propose BackdoorIDS, a simple yet effective zero-shot, inference-time backdoor samples detection method for pretrained vision encoders. BackdoorIDS is motivated by two observations: Attention Hijacking and Restoration. Under progressive input masking, a backdoored image initially concentrates attention on malicious trigger features. Once the masking ratio exceeds the trigger's robustness threshold, the trigger is deactivated, and attention rapidly shifts to benign content. This transition induces a pronounced change in the image embedding, whereas embeddings of clean images evolve more smoothly across masking progress. BackdoorIDS operationalizes this signal by extracting an embedding sequence along the masking trajectory and applying density-based clustering such as DBSCAN. An input is flagged as backdoored if its embedding sequence forms more than one cluster. Extensive experiments show that BackdoorIDS consistently outperforms existing defenses across diverse attack types, datasets, and model families. Notably, it is a plug-and-play approach that requires no retraining and operates fully zero-shot at inference time, making it compatible with a wide range of encoder architectures, including CNNs, ViTs, CLIP, and LLaVA-1.5.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
