对抗攻击已揭示答案：面向视觉语言模型的定向偏差引导测试时防御
Adversarial Attacks Already Tell the Answer: Directional Bias-Guided Test-time Defense for Vision-Language Models
AI总结：
提出定向偏差引导防御（DBD），利用对抗样本在CLIP特征空间中沿主导方向偏移的现象，通过估计防御方向并采用DB分数双流重建策略恢复鲁棒表示，在15个数据集上实现最先进对抗鲁棒性且保持干净准确率。
摘要：
视觉语言模型（VLM），如CLIP，展现出强大的零样本泛化能力，但仍高度易受对抗扰动影响，在现实应用中构成严重风险。针对VLM的测试时防御最近成为一种有前景且高效的方法，无需昂贵的大规模重训练即可防御对抗攻击。在这项工作中，我们发现了一个令人惊讶的现象：在多种输入变换下，CLIP特征空间中的对抗图像始终沿主导方向偏移，而干净图像则呈现分散模式。我们假设这种主导偏移（称为防御方向）与对抗偏移相反，将特征指向正确的类别中心。基于这一见解，我们提出了定向偏差引导防御（DBD），一种测试时框架，用于估计防御方向，并采用基于DB分数的双流重建策略恢复鲁棒表示。在15个数据集上的实验表明，DBD不仅实现了最先进的对抗鲁棒性，同时保持了干净准确率，还揭示了对抗准确率甚至可能超过干净准确率的反直觉结果。这表明对抗扰动内在地编码了关于真实决策边界的定向先验信息。
英文摘要：
Vision-Language Models (VLMs), such as CLIP, have shown strong zero-shot generalization but remain highly vulnerable to adversarial perturbations, posing serious risks in real-world applications. Test-time defenses for VLMs have recently emerged as a promising and efficient approach to defend against adversarial attacks without requiring costly large-scale retraining. In this work, we uncover a surprising phenomenon: under diverse input transformations, adversarial images in CLIP's feature space consistently shift along a dominant direction, in contrast to the dispersed patterns of clean images. We hypothesize that this dominant shift, termed the Defense Direction, opposes the adversarial shift, pointing features back toward their correct class centers. Building on this insight, we propose Directional Bias-guided Defense (DBD), a test-time framework that estimates the Defense Direction and employs a DB-score-based two-stream reconstruction strategy to recover robust representations. Experiments on 15 datasets demonstrate that DBD not only achieves SOTA adversarial robustness while preserving clean accuracy, but also reveals the counterintuitive result that adversarial accuracy can even surpass clean accuracy. This demonstrates that adversarial perturbations inherently encode directional priors about the true decision boundary.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
