ImgTrojan：使用单张图像越狱视觉-语言模型
ImgTrojan: Jailbreaking Vision-Language Models with ONE Image
AI总结：
针对视觉-语言模型安全研究不足的问题，本文提出ImgTrojan，通过训练集中的投毒图像与恶意越狱提示实现单图像越狱，并评估投毒比例、参数位置、成功率与隐蔽性。
摘要：
大语言模型（LLM）与人类价值观的对齐近来受到越来越多关注。然而，LLM与视觉模块集成后形成的视觉-语言模型（VLM）的安全问题仍相对缺乏研究。本文提出一种针对VLM的新型越狱攻击，旨在在用户输入有害指令时绕过其安全屏障。该方法假设训练数据中包含被投毒的（图像，文本）数据对。通过用恶意越狱提示替换原始文本字幕，我们的方法能够借助投毒图像实施越狱攻击。此外，我们分析了投毒比例以及可训练参数位置对攻击成功率的影响。在评估方面，我们设计了两项指标，以量化攻击的成功率和隐蔽性；同时结合一系列精心整理的有害指令，提供了一个用于衡量攻击有效性的基准。通过与基线方法进行比较，我们证明了该攻击的有效性。
英文摘要：
There has been an increasing interest in the alignment of large language models (LLMs) with human values. However, the safety issues of their integration with a vision module, or vision language models (VLMs), remain relatively underexplored. In this paper, we propose a novel jailbreaking attack against VLMs, aiming to bypass their safety barrier when a user inputs harmful instructions. A scenario where our poisoned (image, text) data pairs are included in the training data is assumed. By replacing the original textual captions with malicious jailbreak prompts, our method can perform jailbreak attacks with the poisoned images. Moreover, we analyze the effect of poison ratios and positions of trainable parameters on our attack's success rate. For evaluation, we design two metrics to quantify the success rate and the stealthiness of our attack. Together with a list of curated harmful instructions, a benchmark for measuring attack efficacy is provided. We demonstrate the efficacy of our attack by comparing it with baseline methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
