测量多模态大语言模型中的认知谦逊
Measuring Epistemic Humility in Multimodal Large Language Models
AI总结：
提出HumbleBench基准，通过强制选择多项选择中引入“以上皆非”选项，评估多模态大语言模型拒绝错误选项的谦逊行为。
摘要：
多模态大语言模型（MLLMs）中的幻觉——即模型生成与输入图像不一致的内容——在现实应用中带来显著风险，从视觉问答中的错误信息到决策中的不安全错误。现有基准主要测试识别准确性，即评估模型能否在干扰项中选择正确答案。这忽略了可信AI的另一个重要能力：当没有提供的选项得到图像支持时，能够识别并避免做出错误选择，这是一种与谦逊相关的行为。我们提出了HumbleBench，这是一个新的幻觉基准，旨在评估MLLMs在强制选择多项选择设置中拒绝错误选项的能力，其中包含“以上皆非”选项。基于全景场景图数据集，我们利用对象和关系的细粒度场景图注释，使用候选属性线索，并提示GPT-4-Turbo生成多项选择问题，随后进行严格的人工筛选。每个问题都包含一个“以上皆非”选项，要求模型不仅识别正确的视觉信息，还要识别何时没有提供的答案有效。我们在HumbleBench上评估了各种最先进的MLLMs——包括通用型、专门推理型和专有模型——并为社区报告了实证结果。通过纳入明确的错误选项拒绝，HumbleBench填补了当前评估套件中的一个关键空白，评估了一种较窄但重要的、与可信多模态推理相关的弃权行为。我们的代码和数据集已公开发布，可在https://github.com/maifoundations/HumbleBench获取。
英文摘要：
Hallucinations in multimodal large language models (MLLMs) -- where the model generates content inconsistent with the input image -- pose significant risks in real-world applications, from misinformation in visual question answering to unsafe errors in decision-making. Existing benchmarks primarily test recognition accuracy, i.e., evaluating whether models can select the correct answer among distractors. This overlooks another important capability for trustworthy AI: recognizing when none of the provided options is supported by the image and abstaining from committing to a false choice, a humility-related behavior. We present HumbleBench, a new hallucination benchmark designed to evaluate false-option rejection in MLLMs under a forced-choice multiple-choice setting with a ``None of the above'' option. Built from a panoptic scene graph dataset, we leverage fine-grained scene graph annotations for objects and relations, use candidate attribute cues, and prompt GPT-4-Turbo to generate multiple-choice questions, followed by a rigorous manual filtering process. Each question includes a ``None of the above'' option, requiring models not only to recognize correct visual information but also to identify when no provided answer is valid. We evaluate a variety of state-of-the-art MLLMs -- including general-purpose, specialized reasoning, and proprietary models -- on HumbleBench and report empirical findings for the community. By incorporating explicit false-option rejection, HumbleBench fills a key gap in current evaluation suites by assessing a narrower but important abstention-oriented behavior that is relevant to trustworthy multimodal reasoning. Our code and dataset are released publicly and can be accessed at \href{https://github.com/maifoundations/HumbleBench}{https://github.com/maifoundations/HumbleBench}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
