ODE：多模态大语言模型中幻觉的开放集评估
ODE: Open-Set Evaluation of Hallucinations in Multimodal Large Language Models
AI总结：
本文提出开放集动态评估协议ODE，利用图结构生成多样化样本，在存在性和属性层面评估多模态大语言模型的对象幻觉，揭示数据污染风险并支持幻觉分析与模型微调。
摘要：
幻觉对多模态大语言模型（MLLMs）构成持续挑战。然而，现有用于评估幻觉的基准通常是静态的，可能忽视数据污染的风险。为解决这一问题，我们提出 ODE，一种开放集、动态协议，旨在从存在性和属性两个层面评估 MLLMs 中的对象幻觉。ODE 采用基于图的结构来表示现实世界对象概念、其属性以及它们之间的分布关联。该结构有助于根据多样化的分布标准提取概念组合，生成多样化的样本用于结构化查询，从而在生成式和判别式任务中评估幻觉。通过生成新样本、动态概念组合和变化的分布频率，ODE 降低了数据污染的风险并拓宽了评估范围。该协议适用于通用和专用场景，包括数据有限的场景。实验结果表明了我们协议的有效性，揭示 MLLMs 在使用 ODE 生成的样本进行评估时表现出更高的幻觉率，这表明存在潜在的数据污染。此外，这些生成的样本有助于分析幻觉模式和微调模型，为缓解 MLLMs 中的幻觉提供了一种有效方法。
英文摘要：
Hallucination poses a persistent challenge for multimodal large language models (MLLMs). However, existing benchmarks for evaluating hallucinations are generally static, which may overlook the potential risk of data contamination. To address this issue, we propose ODE, an open-set, dynamic protocol designed to evaluate object hallucinations in MLLMs at both the existence and attribute levels. ODE employs a graph-based structure to represent real-world object concepts, their attributes, and the distributional associations between them. This structure facilitates the extraction of concept combinations based on diverse distributional criteria, generating varied samples for structured queries that evaluate hallucinations in both generative and discriminative tasks. Through the generation of new samples, dynamic concept combinations, and varied distribution frequencies, ODE mitigates the risk of data contamination and broadens the scope of evaluation. This protocol is applicable to both general and specialized scenarios, including those with limited data. Experimental results demonstrate the effectiveness of our protocol, revealing that MLLMs exhibit higher hallucination rates when evaluated with ODE-generated samples, which indicates potential data contamination. Furthermore, these generated samples aid in analyzing hallucination patterns and fine-tuning models, offering an effective approach to mitigating hallucinations in MLLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
