arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.03554cs.CLcs.CRcs.LG
收藏

针对通过视觉提示注入实现目标劫持的大型视觉语言模型的实证分析

Empirical Analysis of Large Vision-Language Models against Goal Hijacking via Visual Prompt Injection

Tohoku University(东北大学);NTT Human Informatics Laboratories, NTT Corporation(NTT公司NTT人类信息学实验室)

Subaru Kimura, Ryota Tanaka, Shumpei Miyawaki, Jun Suzuki, Keisuke Sakaguchi

AI总结:

本文提出视觉提示注入攻击方法GHVPI,可劫持大型视觉语言模型的执行任务,实验显示GPT-4V攻击成功率达15.8%,揭示了其安全风险。

摘要:

我们探索了视觉提示注入(VPI),该方法恶意利用大型视觉语言模型(LVLMs)遵循绘制在输入图像上的指令的能力。我们提出了一种新的VPI方法,名为“通过视觉提示注入实现目标劫持”(GHVPI),该方法将LVLMs的执行任务从原始任务切换为攻击者指定的替代任务。定量分析表明,GPT-4V容易受到GHVPI的攻击,并表现出显著的15.8%的攻击成功率,这是一个不可忽视的安全风险。我们的分析还表明,成功的GHVPI需要LVLMs具备较高的字符识别能力和指令遵循能力。

英文摘要:

We explore visual prompt injection (VPI) that maliciously exploits the ability of large vision-language models (LVLMs) to follow instructions drawn onto the input image. We propose a new VPI method, "goal hijacking via visual prompt injection" (GHVPI), that swaps the execution task of LVLMs from an original task to an alternative task designated by an attacker. The quantitative analysis indicates that GPT-4V is vulnerable to the GHVPI and demonstrates a notable attack success rate of 15.8%, which is an unignorable security risk. Our analysis also shows that successful GHVPI requires high character recognition capability and instruction-following ability in LVLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑