欺骗你的多模态大语言模型有多容易?基于欺骗性提示的实证分析
How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Prompts
AI总结:
本文提出MAD-Bench基准评估多模态大语言模型对欺骗性提示的脆弱性,发现GPT-4o准确率达82.82%远超其他模型(9%-50%),并提出通过添加提示段落鼓励模型三思的简单方法使准确率翻倍。
摘要:
多模态大语言模型(MLLMs)的显著进展并未使其免受挑战,尤其是在处理提示中的欺骗性信息时,会在此类条件下产生幻觉响应。为了定量评估这种脆弱性,我们提出了MAD-Bench,这是一个精心策划的基准,包含1000个测试样本,分为5个类别,如不存在的对象、对象计数和空间关系。我们对流行的MLLMs进行了全面分析,涵盖从GPT-4v、Reka、Gemini-Pro到开源模型(如LLaVA-NeXT和MiniCPM-Llama3)。实验中,我们观察到GPT-4o与其他模型之间存在显著的性能差距;且之前的鲁棒指令微调模型在这一新基准上并不有效。GPT-4o在MAD-Bench上达到82.82%的准确率,而其他模型的准确率范围为9%至50%。我们进一步提出一种补救方法,在欺骗性提示中添加额外段落,鼓励模型在回答问题前三思。令人惊讶的是,这种简单方法甚至能使准确率翻倍;然而,绝对数值仍过低,难以令人满意。我们希望MAD-Bench能作为有价值的基准,推动进一步研究以增强模型对欺骗性提示的鲁棒性。
英文摘要:
The remarkable advancements in Multimodal Large Language Models (MLLMs) have not rendered them immune to challenges, particularly in the context of handling deceptive information in prompts, thus producing hallucinated responses under such conditions. To quantitatively assess this vulnerability, we present MAD-Bench, a carefully curated benchmark that contains 1000 test samples divided into 5 categories, such as non-existent objects, count of objects, and spatial relationship. We provide a comprehensive analysis of popular MLLMs, ranging from GPT-4v, Reka, Gemini-Pro, to open-sourced models, such as LLaVA-NeXT and MiniCPM-Llama3. Empirically, we observe significant performance gaps between GPT-4o and other models; and previous robust instruction-tuned models are not effective on this new benchmark. While GPT-4o achieves 82.82% accuracy on MAD-Bench, the accuracy of any other model in our experiments ranges from 9% to 50%. We further propose a remedy that adds an additional paragraph to the deceptive prompts to encourage models to think twice before answering the question. Surprisingly, this simple method can even double the accuracy; however, the absolute numbers are still too low to be satisfactory. We hope MAD-Bench can serve as a valuable benchmark to stimulate further research to enhance model resilience against deceptive prompts.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。