CrossCheckGPT：多模态基础模型的通用幻觉排名
CrossCheckGPT: Universal Hallucination Ranking for Multimodal Foundation Models
AI总结：
针对多模态基础模型幻觉评估，提出无参考的通用排名方法CrossCheckGPT，利用跨系统一致性打分，适用于任意模型和任务，并在文本、图像及音视频基准上验证，与人类判断高度相关。
摘要：
多模态基础模型容易产生幻觉，生成与输入相矛盾或缺乏事实依据的输出。由于架构、训练数据和指令微调技术的多样性，系统对幻觉的敏感性可能存在很大差异。为了评估系统的幻觉鲁棒性，已针对图像描述、问答、摘要或传记生成等特定任务开发了幻觉排名方法。然而，这些方法通常将模型输出与黄金标准参考或标签进行比较，限制了新领域的幻觉基准测试。本工作提出了“CrossCheckGPT”，一种用于多模态基础模型的无参考通用幻觉排名方法。CrossCheckGPT的核心思想是，相同的幻觉内容不太可能由不同的独立系统生成，因此跨系统一致性可以提供有意义且准确的幻觉评估分数。CrossCheckGPT可应用于任何模型或任务，前提是通过适当的距离度量可以衡量输出之间的信息一致性。针对生成文本的多模态大语言模型，我们探索了两种信息一致性度量：CrossCheck-explicit和CrossCheck-implicit。我们展示了我们的方法在文本、图像和音视频领域等多种模态的幻觉排名中的适用性。此外，我们提出了首个音视频幻觉基准“AVHalluBench”，并展示了CrossCheckGPT的有效性，在MHaluBench和AVHalluBench上分别实现了与人类判断98%和89%的相关性。
英文摘要：
Multimodal foundation models are prone to hallucination, generating outputs that either contradict the input or are not grounded by factual information. Given the diversity in architectures, training data and instruction tuning techniques, there can be large variations in systems' susceptibility to hallucinations. To assess system hallucination robustness, hallucination ranking approaches have been developed for specific tasks such as image captioning, question answering, summarization, or biography generation. However, these approaches typically compare model outputs to gold-standard references or labels, limiting hallucination benchmarking for new domains. This work proposes "CrossCheckGPT", a reference-free universal hallucination ranking for multimodal foundation models. The core idea of CrossCheckGPT is that the same hallucinated content is unlikely to be generated by different independent systems, hence cross-system consistency can provide meaningful and accurate hallucination assessment scores. CrossCheckGPT can be applied to any model or task, provided that the information consistency between outputs can be measured through an appropriate distance metric. Focusing on multimodal large language models that generate text, we explore two information consistency measures: CrossCheck-explicit and CrossCheck-implicit. We showcase the applicability of our method for hallucination ranking across various modalities, namely the text, image, and audio-visual domains. Further, we propose the first audio-visual hallucination benchmark, "AVHalluBench", and illustrate the effectiveness of CrossCheckGPT, achieving correlations of 98% and 89% with human judgements on MHaluBench and AVHalluBench, respectively.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
