CaptionFool: 针对最新Transformer图像描述模型的通用图像描述模型攻击
CaptionFool: Universal Image Captioning Model Attacks
AI总结：
CaptionFool通过修改少量图像块，成功生成任意目标描述，揭示了视觉-语言模型的安全漏洞。
摘要：
图像描述模型是基于大规模图像-文本数据集训练的编码器-解码器架构，使其容易受到对抗性攻击。我们提出了CaptionFool，一种新颖的通用（输入无关）对抗性攻击方法，针对最新Transformer基于的图像描述模型。通过修改仅577个图像块中的7个（大约1.2%的图像），我们的攻击在生成任意目标描述中实现了94-96%的成功率，包括具有攻击性的内容。我们进一步证明，CaptionFool可以生成专门设计以逃避现有内容审核过滤器的“俚语”术语。我们的发现揭示了部署的视觉-语言模型中的关键漏洞，并强调了对这类攻击进行稳健防御的紧迫需求。警告：本文包含模型输出，这些输出具有攻击性。
英文摘要：
Image captioning models are encoder-decoder architectures trained on large-scale image-text datasets, making them susceptible to adversarial attacks. We present CaptionFool, a novel universal (input-agnostic) adversarial attack against state-of-the-art transformer-based captioning models. By modifying only 7 out of 577 image patches (approximately 1.2% of the image), our attack achieves 94-96% success rate in generating arbitrary target captions, including offensive content. We further demonstrate that CaptionFool can generate "slang" terms specifically designed to evade existing content moderation filters. Our findings expose critical vulnerabilities in deployed vision-language models and underscore the urgent need for robust defenses against such attacks. Warning: This paper contains model outputs which are offensive in nature.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
