arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.01449cs.CRcs.AI
收藏

VisInject:破坏≠注入——对视觉-语言模型中通用对抗攻击的双维度评估

VisInject: Disruption != Injection -- A Dual-Dimension Evaluation of Universal Adversarial Attacks on Vision-Language Models

Department of Electrical and Computer Engineering, Tufts University(Tufts大学电气与计算机工程系)

Pang Liu, Yingjie Lao

AI总结:

本文通过双维度评估揭示视觉-语言模型中通用对抗攻击的脆弱性,发现攻击成功与精确注入存在显著差异,揭示了模型在微小扰动下的表现。

摘要:

针对对齐的多模态大语言模型中的通用对抗攻击,报告了攻击成功率在60-80%之间,表明视觉模态极易受到不可察觉扰动的影响作为提示注入渠道。我们论证该数字混淆了两个不同事件:(i)模型输出被扰动(影响)和(ii)攻击者所选的目标概念实际上被发出(精确注入)。我们采用两种现有技术——通用对抗攻击和AnyAttack——在$L_{inf}$预算为16/255下进行评估,并添加双轴评估:确定性Ratcliff-Obershelp漂移分数用于影响(程序化基线)加上四级有序类别非/弱/部分/确认用于精确注入。评判者为DeepSeek-V4-Pro,在思考模式下校准,与Claude Opus 4.7在注入轴上Cohen's $κ$ = 0.77(显著一致);整个4475条SHA-256输入缓存随数据集一起提供,使评审员可以重新推导论文编号而不需API密钥。在6615对数据、四个开放VLMs、七个攻击提示和七个测试图像上,两个轴相差约90倍:66.4%的对被程序性扰动(LLM判断46.6%在显著或完整级别),但仅有0.756%(50/6615)达到任何非none注入级别,仅0.030%(2/6615)精确注入。少数成功注入的案例集中在截图或文档风格载体上,其语义本身已邀请文本转录。BLIP-2在$L_{inf}$=16/255下,所有2205对数据上均未显示可检测漂移,即使作为Stage-1替代品使用。我们发布了完整数据集——21张通用图像、147张对抗性照片、6,615对响应、v3双轴评判结果和缓存,网址为huggingface.co/datasets/jeffliulab/visinject。

英文摘要:

Universal adversarial attacks on aligned multimodal large language models are increasingly reported with attack success rates in the 60-80% range, suggesting the visual modality is highly vulnerable to imperceptible perturbations as a prompt-injection channel. We argue that this number conflates two distinct events: (i) the model's output was perturbed (Influence), and (ii) the attacker's chosen target concept was actually emitted (Precise Injection). We compose two existing techniques -- Universal Adversarial Attack and AnyAttack -- under an $L_{inf}$ budget of 16/255, and we add a dual-axis evaluation: a deterministic Ratcliff-Obershelp drift score for Influence (programmatic baseline) plus a 4-tier ordinal categorical none/weak/partial/confirmed for Precise Injection. The judge is DeepSeek-V4-Pro in thinking mode, calibrated against Claude Opus 4.7 with Cohen's $κ$ = 0.77 on the injection axis (substantial agreement); the entire 4475-entry SHA-256 input cache ships with the dataset so reviewers can re-derive paper numbers bit-exact without an API key. Across 6615 pairs over four open VLMs, seven attack prompts, and seven test images, the two axes diverge by roughly 90$\times$: 66.4% of pairs are programmatically disturbed (LLM-judged 46.6% at the substantial-or-complete tier), but only 0.756% (50/6615) reach any non-none injection tier and only 0.030% (2/6615) verbatim. The few injections that do land cluster on screenshot- or document-style carriers whose semantics already invite text transcription. BLIP-2 shows \emph{zero detectable drift} at $L_{inf}$ = 16/255 across all 2205 pairs even when used as a Stage-1 surrogate. We release the full dataset -- 21 universal images, 147 adversarial photos, 6,615 response pairs, the v3 dual-axis judge results, and the cache at huggingface.co/datasets/jeffliulab/visinject.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑