AMBER：一个无需LLM的多维基准，用于MLLMs幻觉评估
AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation
AI总结：
本文提出无需LLM的多维基准AMBER，覆盖生成与判别任务中的存在性、属性和关系幻觉评估，并设计低成本评估流程，对主流MLLMs进行系统分析并提出缓解幻觉建议。
摘要：
尽管在多模态任务中取得了显著进展，当前的多模态大语言模型（MLLMs）仍面临幻觉这一重大挑战，幻觉可能导致有害后果。因此，评估MLLMs的幻觉在模型改进和实际应用部署中变得越来越重要。以往的工作受限于高昂的评估成本（例如依赖人类或先进LLMs）和不足的评估维度（例如任务类型和幻觉类型）。在本文中，我们提出了一个无需LLM的多维基准AMBER，可用于评估生成式任务和判别式任务，包括存在性、属性和关系幻觉。基于AMBER，我们设计了一个低成本且高效的评估流程。此外，我们对主流MLLMs（包括GPT-4V(ision)）进行了全面评估和详细分析，并给出了缓解幻觉的指导性建议。AMBER的数据和代码可在 https://github.com/junyangwang0410/AMBER 获取。
英文摘要：
Despite making significant progress in multi-modal tasks, current Multi-modal Large Language Models (MLLMs) encounter the significant challenge of hallucinations, which may lead to harmful consequences. Therefore, evaluating MLLMs' hallucinations is becoming increasingly important in model improvement and practical application deployment. Previous works are limited in high evaluation costs (e.g., relying on humans or advanced LLMs) and insufficient evaluation dimensions (e.g., types of tasks and hallucinations). In this paper, we propose an LLM-free multi-dimensional benchmark AMBER, which can be used to evaluate both generative task and discriminative task including existence, attribute and relation hallucination. Based on AMBER, we design a low-cost and efficient evaluation pipeline. Additionally, we conduct a comprehensive evaluation and detailed analysis of mainstream MLLMs including GPT-4V(ision), and also give guideline suggestions for mitigating hallucinations. The data and code of AMBER are available at https://github.com/junyangwang0410/AMBER.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
