arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.02889cs.CLcs.LG
收藏

InterrogateLLM:大语言模型生成答案中的零资源幻觉检测

InterrogateLLM: Zero-Resource Hallucination Detection in LLM-Generated Answers

Microsoft(微软);Technion(以色列理工学院);The Open University(开放大学);Tel-Aviv University(特拉维夫大学)

Yakir Yehuda, Itzik Malkiel, Oren Barkan, Jonathan Weill, Royi Ronen, Noam Koenigstein

AI总结:

针对大语言模型的幻觉问题,本文提出零资源幻觉检测方法InterrogateLLM,无需外部知识即可有效检测幻觉,在多模型多数据集评估中表现优异,其中对Llama-2的幻觉检测平衡准确率达81%。

摘要:

尽管大语言模型(LLM)已取得诸多进展且发展速度前所未有,但受多种因素限制,其对日常生活各方面的影响与融入程度仍有限。阻碍其广泛应用的一个关键因素是幻觉问题,即LLM生成的答案听起来合理,却与事实真相不符。本文提出一种检测大语言模型幻觉的新方法,解决了这类模型在各类真实场景落地中的关键问题。通过在多个数据集和包括Llama-2在内的多种LLM上开展广泛评估,我们研究了近年多款LLM的幻觉水平,并证明了该方法自动检测幻觉的有效性。值得注意的是,在一项特定实验中,我们观察到Llama-2的幻觉占比高达87%,而我们的方法在完全不依赖外部知识的情况下,取得了81%的平衡准确率。

英文摘要:

Despite the many advances of Large Language Models (LLMs) and their unprecedented rapid evolution, their impact and integration into every facet of our daily lives is limited due to various reasons. One critical factor hindering their widespread adoption is the occurrence of hallucinations, where LLMs invent answers that sound realistic, yet drift away from factual truth. In this paper, we present a novel method for detecting hallucinations in large language models, which tackles a critical issue in the adoption of these models in various real-world scenarios. Through extensive evaluations across multiple datasets and LLMs, including Llama-2, we study the hallucination levels of various recent LLMs and demonstrate the effectiveness of our method to automatically detect them. Notably, we observe up to 87% hallucinations for Llama-2 in a specific experiment, where our method achieves a Balanced Accuracy of 81%, all without relying on external knowledge.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑