SelfCheckGPT：生成式大语言模型的零资源黑盒幻觉检测
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models
AI总结：
提出SelfCheckGPT，一种基于采样的零资源黑盒方法，通过检测采样响应的一致性来识别大语言模型的幻觉，在句子级检测和段落级评估中优于灰盒基线。
摘要：
生成式大语言模型（LLM）如GPT-3能够对各种各样的用户提示生成高度流畅的响应。然而，已知LLM会幻觉化事实并做出非事实性陈述，这可能削弱对其输出的信任。现有的事实核查方法要么需要访问输出概率分布（对于像ChatGPT这样的系统可能不可用），要么需要外部数据库，这些数据库通过独立的、通常复杂的模块进行接口。在这项工作中，我们提出了“SelfCheckGPT”，一种简单的基于采样的方法，可用于以零资源方式（即无需外部数据库）对黑盒模型的响应进行事实核查。SelfCheckGPT利用了一个简单的想法：如果LLM对给定概念有知识，采样的响应可能相似并包含一致的事实。然而，对于幻觉化的事实，随机采样的响应可能发散并相互矛盾。我们通过使用GPT-3从WikiBio数据集生成关于个人的段落，并手动标注生成段落的事实性来研究这种方法。我们证明SelfCheckGPT可以：i)检测非事实性和事实性句子；ii)按事实性对段落进行排序。我们将我们的方法与几个基线进行比较，并表明与灰盒方法相比，我们的方法在句子级幻觉检测中具有显著更高的AUC-PR分数，在段落级事实性评估中具有更高的相关性分数。
英文摘要：
Generative Large Language Models (LLMs) such as GPT-3 are capable of generating highly fluent responses to a wide variety of user prompts. However, LLMs are known to hallucinate facts and make non-factual statements which can undermine trust in their output. Existing fact-checking approaches either require access to the output probability distribution (which may not be available for systems such as ChatGPT) or external databases that are interfaced via separate, often complex, modules. In this work, we propose "SelfCheckGPT", a simple sampling-based approach that can be used to fact-check the responses of black-box models in a zero-resource fashion, i.e. without an external database. SelfCheckGPT leverages the simple idea that if an LLM has knowledge of a given concept, sampled responses are likely to be similar and contain consistent facts. However, for hallucinated facts, stochastically sampled responses are likely to diverge and contradict one another. We investigate this approach by using GPT-3 to generate passages about individuals from the WikiBio dataset, and manually annotate the factuality of the generated passages. We demonstrate that SelfCheckGPT can: i) detect non-factual and factual sentences; and ii) rank passages in terms of factuality. We compare our approach to several baselines and show that our approach has considerably higher AUC-PR scores in sentence-level hallucination detection and higher correlation scores in passage-level factuality assessment compared to grey-box methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
