重新审视CroPA：面向视觉-语言模型跨提示对抗转移性的可重复性研究与改进
Revisiting CroPA: A Reproducibility Study and Enhancements for Cross-Prompt Adversarial Transferability in Vision-Language Models
AI总结：
本文重新审视CroPA，验证其跨提示转移性，并提出改进方法，包括新的初始化策略、跨图像迁移性研究及针对视觉编码器的损失函数，提升对抗有效性。
摘要：
大型视觉-语言模型（VLMs）已革新计算机视觉，使图像分类、描述生成和视觉问答等任务成为可能。然而，它们仍高度易受对抗攻击影响，特别是在视觉和文本模态均可操控的情况下。在本研究中，我们对“一张图片等于一千个谎言：在视觉-语言模型上跨提示的对抗转移性”进行了全面的可重复性研究，验证了跨提示攻击（CroPA）并确认其优于现有基线的跨提示转移性。除了复制外，我们还提出了几个关键改进：（1）一种新的初始化策略，显著提高了攻击成功率（ASR）。（2）通过学习通用扰动来研究跨图像转移性。（3）一种针对视觉编码器注意力机制的新型损失函数以提高泛化能力。我们的评估涵盖主流VLMs，包括Flamingo、BLIP-2和InstructBLIP，以及在LLaVA上的扩展实验，验证了原始结果并证明我们的改进一致提升了对抗有效性。我们的工作强调了研究VLMs对抗脆弱性的重要性，并提供了一个更稳健的框架来生成可迁移的对抗示例，对理解VLMs在现实应用中的安全性具有重要意义。
英文摘要：
Large Vision-Language Models (VLMs) have revolutionized computer vision, enabling tasks such as image classification, captioning, and visual question answering. However, they remain highly vulnerable to adversarial attacks, particularly in scenarios where both visual and textual modalities can be manipulated. In this study, we conduct a comprehensive reproducibility study of "An Image is Worth 1000 Lies: Adversarial Transferability Across Prompts on Vision-Language Models" validating the Cross-Prompt Attack (CroPA) and confirming its superior cross-prompt transferability compared to existing baselines. Beyond replication we propose several key improvements: (1) A novel initialization strategy that significantly improves Attack Success Rate (ASR). (2) Investigate cross-image transferability by learning universal perturbations. (3) A novel loss function targeting vision encoder attention mechanisms to improve generalization. Our evaluation across prominent VLMs -- including Flamingo, BLIP-2, and InstructBLIP as well as extended experiments on LLaVA validates the original results and demonstrates that our improvements consistently boost adversarial effectiveness. Our work reinforces the importance of studying adversarial vulnerabilities in VLMs and provides a more robust framework for generating transferable adversarial examples, with significant implications for understanding the security of VLMs in real-world applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
