arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.14564cs.CL
收藏

语言模型会产生幻觉,但可能在事实验证方面表现出色

Language Models Hallucinate, but May Excel at Fact Verification

Allen Institute for AI(艾伦人工智能研究所);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Tsinghua University(清华大学)

Jian Guan, Jesse Dodge, David Wadden, Minlie Huang, Hao Peng

AI总结:

针对大型语言模型严重的幻觉问题,本研究通过人工评估证实其可被重新用作有效的事实验证器,并发现FLAN-T5-11B表现最佳,为开发可信生成模型提供了见解。

摘要:

自然语言处理(NLP)领域的最新进展很大程度上归功于大型语言模型(LLMs)的显著进步。然而,LLMs经常“产生幻觉”,导致输出不符合事实。我们精心设计的人工评估证实了严重的幻觉问题,揭示出即使是GPT-3.5,产生符合事实输出的比例也不到25%。这凸显了事实验证器的重要性,以便衡量和激励进展。我们的系统调查证实,LLMs可被重新用作有效的事实验证器,且与人类判断具有强相关性。令人惊讶的是,FLAN-T5-11B,即我们研究中事实性最差的生成器,作为事实验证器的表现却最佳,甚至超越了GPT3.5和ChatGPT等更强大的LLMs。进一步深入分析,我们研究了这些LLMs对高质量证据的依赖,以及它们在鲁棒性和泛化能力上的缺陷。我们的研究为开发可信的生成模型提供了见解。

英文摘要:

Recent progress in natural language processing (NLP) owes much to remarkable advances in large language models (LLMs). Nevertheless, LLMs frequently "hallucinate," resulting in non-factual outputs. Our carefully-designed human evaluation substantiates the serious hallucination issue, revealing that even GPT-3.5 produces factual outputs less than 25% of the time. This underscores the importance of fact verifiers in order to measure and incentivize progress. Our systematic investigation affirms that LLMs can be repurposed as effective fact verifiers with strong correlations with human judgments. Surprisingly, FLAN-T5-11B, the least factual generator in our study, performs the best as a fact verifier, even outperforming more capable LLMs like GPT3.5 and ChatGPT. Delving deeper, we analyze the reliance of these LLMs on high-quality evidence, as well as their deficiencies in robustness and generalization ability. Our study presents insights for developing trustworthy generation models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑