当数据操纵遇上攻击目标：VLMs攻击的深度综述
When Data Manipulation Meets Attack Goals: An In-depth Survey of Attacks for VLMs
AI总结：
本文针对视觉语言模型（VLMs）的安全漏洞，按攻击目标分类综述其定制化攻击策略及数据操纵方法，提出攻击分类法，总结评估指标并讨论未来研究方向，以增强VLMs的鲁棒性和安全性。
摘要：
近年来，视觉语言模型（VLMs）因能有效整合和处理文本与视觉信息的卓越能力而备受关注。这种整合显著提升了场景感知、机器人技术等各类应用的性能。然而，VLMs的部署也引发了严重的安全问题，需大量研究评估其潜在漏洞。本文对VLMs的定制化攻击策略进行深度综述，按攻击目标（越狱、伪装、利用）分类，同时详细介绍VLMs数据操纵的各类方法，还概述了缓解这些漏洞的防御机制。通过识别不同攻击类型的关键联系与区别，提出VLMs攻击的分类法，总结全面描述不同攻击特征及对VLMs影响的评估指标。最后讨论能进一步增强VLMs鲁棒性和安全性的未来研究方向，强调该领域持续探索的重要性。为促进社区参与，我们维护一个更新的项目页面：https://github.com/AobtDai/VLM_Attack_Paper_List。
英文摘要：
Vision-Language Models (VLMs) have gained considerable prominence in recent years due to their remarkable capability to effectively integrate and process both textual and visual information. This integration has significantly enhanced performance across a diverse spectrum of applications, such as scene perception and robotics. However, the deployment of VLMs has also given rise to critical safety and security concerns, necessitating extensive research to assess the potential vulnerabilities these VLM systems may harbor. In this work, we present an in-depth survey of the attack strategies tailored for VLMs. We categorize these attacks based on their underlying objectives - namely jailbreak, camouflage, and exploitation - while also detailing the various methodologies employed for data manipulation of VLMs. Meanwhile, we outline corresponding defense mechanisms that have been proposed to mitigate these vulnerabilities. By discerning key connections and distinctions among the diverse types of attacks, we propose a compelling taxonomy for VLM attacks. Moreover, we summarize the evaluation metrics that comprehensively describe the characteristics and impact of different attacks on VLMs. Finally, we conclude with a discussion of promising future research directions that could further enhance the robustness and safety of VLMs, emphasizing the importance of ongoing exploration in this critical area of study. To facilitate community engagement, we maintain an up-to-date project page, accessible at: https://github.com/AobtDai/VLM_Attack_Paper_List.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
