大型语言模型中幻觉缓解技术的全面综述
A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models
AI总结：
本文全面综述了32种以上缓解大型语言模型幻觉的技术，引入基于数据集利用、任务、反馈机制等参数的分类法，并分析现有技术的局限，为未来研究奠定基础。
摘要：
随着大型语言模型（LLMs）在生成类人文本能力上的不断提升，一个关键挑战仍然存在，即它们倾向于产生幻觉，生成看似符合事实但毫无根据的内容。幻觉问题可以说是将这些强大的LLMs安全部署到影响人们生活的实际生产系统中的最大障碍。在实际环境中广泛采用LLMs的进程，在很大程度上依赖于对幻觉的解决和缓解。与专注于有限任务的传统AI系统不同，LLMs在训练期间接触了海量的在线文本数据。虽然这使它们展现出令人印象深刻的语言流畅性，但也意味着它们能够从训练数据的偏差中推断信息、误解有歧义的提示，或修改信息以在表面上与输入保持一致。当我们依赖语言生成能力来处理敏感应用时，如总结医疗记录、财务分析报告等，这就变得极其令人担忧。本文对为缓解LLMs幻觉而开发的32种以上技术进行了全面综述。其中值得注意的是检索增强生成（Retrieval Augmented Generation，Lewis等人，2021）、知识检索（Knowledge Retrieval，Varshney等人，2023）、CoNLI（Lei等人，2023）和CoVe（Dhuliawala等人，2023）。此外，我们引入了一个详细的分类法，根据数据集利用、常见任务、反馈机制和检索器类型等各种参数对这些方法进行分类。这种分类有助于区分专门为解决LLMs幻觉问题而设计的各种方法。另外，我们分析了这些技术固有的挑战和局限性，为未来在LLMs领域内解决幻觉及相关现象的研究提供了坚实的基础。
英文摘要：
As Large Language Models (LLMs) continue to advance in their ability to write human-like text, a key challenge remains around their tendency to hallucinate generating content that appears factual but is ungrounded. This issue of hallucination is arguably the biggest hindrance to safely deploying these powerful LLMs into real-world production systems that impact people's lives. The journey toward widespread adoption of LLMs in practical settings heavily relies on addressing and mitigating hallucinations. Unlike traditional AI systems focused on limited tasks, LLMs have been exposed to vast amounts of online text data during training. While this allows them to display impressive language fluency, it also means they are capable of extrapolating information from the biases in training data, misinterpreting ambiguous prompts, or modifying the information to align superficially with the input. This becomes hugely alarming when we rely on language generation capabilities for sensitive applications, such as summarizing medical records, financial analysis reports, etc. This paper presents a comprehensive survey of over 32 techniques developed to mitigate hallucination in LLMs. Notable among these are Retrieval Augmented Generation (Lewis et al, 2021), Knowledge Retrieval (Varshney et al,2023), CoNLI (Lei et al, 2023), and CoVe (Dhuliawala et al, 2023). Furthermore, we introduce a detailed taxonomy categorizing these methods based on various parameters, such as dataset utilization, common tasks, feedback mechanisms, and retriever types. This classification helps distinguish the diverse approaches specifically designed to tackle hallucination issues in LLMs. Additionally, we analyze the challenges and limitations inherent in these techniques, providing a solid foundation for future research in addressing hallucinations and related phenomena within the realm of LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
