一针及时省九针：通过验证低置信度生成来检测和缓解大语言模型的幻觉
A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation
AI总结：
本文提出利用模型logit输出值识别低置信度候选内容，通过验证程序主动检测并缓解大语言模型生成过程中的幻觉，在GPT-3.5上使幻觉率从47.5%降至14.5%，并验证了跨任务和跨模型的有效性。
摘要：
近年来发展起来的大语言模型在生成流畅且连贯的文本方面取得了显著成功。然而，这些模型往往倾向于产生“幻觉”，这严重损害了它们的可靠性。在这项工作中，我们针对这一关键问题提出了一种在生成过程中主动检测和缓解幻觉的方法。具体来说，我们首先利用模型的logit输出值识别潜在幻觉的候选内容，通过验证程序检查其正确性，缓解检测到的幻觉，然后继续生成过程。通过在GPT-3.5（text-davinci-003）上对“文章生成任务”进行的大量实验，我们首先证明了检测和缓解技术各自的功效。具体而言，检测技术实现了约88%的召回率，缓解技术成功缓解了57.6%的正确检测到的幻觉。重要的是，即使在错误检测到的幻觉（即假阳性）情况下，我们的缓解技术也不会引入新的幻觉。然后，我们表明所提出的主动检测和缓解方法成功地将GPT-3.5模型的幻觉率从平均47.5%降低到14.5%。我们进一步通过额外研究证明了该方法的有效性和广泛适用性，包括在不同类型问题（多跳和虚假前提问题）上的表现，以及使用来自不同模型家族的另一个大语言模型（Vicuna）进行测试。总之，我们的工作有助于提高大语言模型的可靠性和可信度，这是使其在现实世界应用中广泛采用的关键一步。
英文摘要：
Recently developed large language models have achieved remarkable success in generating fluent and coherent text. However, these models often tend to 'hallucinate' which critically hampers their reliability. In this work, we address this crucial problem and propose an approach that actively detects and mitigates hallucinations during the generation process. Specifically, we first identify the candidates of potential hallucination leveraging the model's logit output values, check their correctness through a validation procedure, mitigate the detected hallucinations, and then continue with the generation process. Through extensive experiments with GPT-3.5 (text-davinci-003) on the 'article generation task', we first demonstrate the individual efficacy of our detection and mitigation techniques. Specifically, the detection technique achieves a recall of ~88% and the mitigation technique successfully mitigates 57.6% of the correctly detected hallucinations. Importantly, our mitigation technique does not introduce new hallucinations even in the case of incorrectly detected hallucinations, i.e., false positives. Then, we show that the proposed active detection and mitigation approach successfully reduces the hallucinations of the GPT-3.5 model from 47.5% to 14.5% on average. We further demonstrate the effectiveness and wide applicability of our approach through additional studies including performance on different types of questions (multi-hop and false premise questions) and with another LLM from a different model family (Vicuna). In summary, our work contributes to improving the reliability and trustworthiness of large language models, a crucial step en route to enabling their widespread adoption in real-world applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
