arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2312.07553cs.AIcs.CL
收藏

大型多模态模型中的上下文劫持

Hijacking Context in Large Multi-modal Models

NAVER Cloud(NAVER云);Korea Advanced Institute of Science Technology (KAIST)(韩国科学技术院)

Joonhyun Jeong

AI总结:

本研究发现现成大型多模态模型存在易被少量不连贯图文误导的上下文劫持问题,提出基于GPT-4V的预过滤方法移除无关上下文,并探究替换被劫持上下文以生成连贯响应的可行性。

摘要:

近年来,大型多模态模型(LMMs)已展现出在给定图像相关指令的情况下理解图像视觉内容的能力。基于大语言模型(LLMs)构建的LMMs也继承了其能力与特性,例如上下文学习——即输入提示为连贯的图像与文本序列。然而,我们发现了现成LMMs的一项新局限:一小部分不连贯的图像或文本描述会误导LMMs,使其仅生成关于被劫持上下文的有偏输出,而非原本预期的上下文。为解决这一问题,我们提出了一种预过滤方法,利用GPT-4V对上下文内分布偏移的鲁棒性,通过GPT-4V移除无关上下文。我们进一步探究了借助GPT-4V和文本生成图像模型,将被劫持的视觉与文本上下文替换为相关内容,是否有助于生成连贯的响应。

英文摘要:

Recently, Large Multi-modal Models (LMMs) have demonstrated their ability to understand the visual contents of images given the instructions regarding the images. Built upon the Large Language Models (LLMs), LMMs also inherit their abilities and characteristics such as in-context learning where a coherent sequence of images and texts are given as the input prompt. However, we identify a new limitation of off-the-shelf LMMs where a small fraction of incoherent images or text descriptions mislead LMMs to only generate biased output about the hijacked context, not the originally intended context. To address this, we propose a pre-filtering method that removes irrelevant contexts via GPT-4V, based on its robustness towards distribution shift within the contexts. We further investigate whether replacing the hijacked visual and textual contexts with the correlated ones via GPT-4V and text-to-image models can help yield coherent responses.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑