视觉-语言模型中的对抗防御：概述
Adversarial Defense in Vision-Language Models: An Overview
AI总结：
本文综述了视觉-语言模型对抗防御的最新进展，探讨了三种主要防御范式及其优缺点，旨在提升模型的鲁棒性。
摘要：
视觉语言模型（VLMs，例如CLIP）的广泛应用引发了对其对复杂且难以察觉的对抗性攻击的脆弱性的担忧。这些攻击可能在跨模态任务中破坏模型性能和系统安全。为了解决这一挑战，提出了三种主要的防御范式：训练时防御、测试时适应防御和无训练防御。训练时防御涉及修改训练过程，通常通过对抗性微调来提高对对抗示例的鲁棒性。虽然有效，但这种方法需要大量的计算资源，并且可能无法在所有对抗攻击上泛化。测试时适应防御专注于在推理时通过更新参数来处理未标记的对抗示例，提供了灵活性，但通常以增加复杂性和计算开销为代价。无训练防御避免修改模型本身，而是专注于改变对抗性输入或其特征嵌入，这会强制输入扰动以减轻攻击的影响，而无需额外训练。本文综述了VLMs对抗防御策略的最新进展，突显了这些方法的优缺点，并讨论了增强VLMs鲁棒性的持续挑战。
英文摘要：
The widespread use of Vision Language Models (VLMs, e.g. CLIP) has raised concerns about their vulnerability to sophisticated and imperceptible adversarial attacks. These attacks could compromise model performance and system security in cross-modal tasks. To address this challenge, three main defense paradigms have been proposed: Training-time Defense, Test-time Adaptation Defense, and Training-free Defense. Training-time Defense involves modifying the training process, typically through adversarial fine-tuning to improve the robustness to adversarial examples. While effective, this approach requires substantial computational resources and may not generalize across all adversarial attacks. Test-time Adaptation Defense focuses on adapting the model at inference time by updating its parameters to handle unlabeled adversarial examples, offering flexibility but often at the cost of increased complexity and computational overhead. Training-free Defense avoids modifying the model itself, instead focusing on altering the adversarial inputs or their feature embeddings, which enforces input perturbations to mitigate the impact of attacks without additional training. This survey reviews the latest advancements in adversarial defense strategies for VLMs, highlighting the strengths and limitations of such approaches and discussing ongoing challenges in enhancing the robustness of VLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
