大型语言模型的妄想
Delusions of Large Language Models
AI总结：
本文提出LLM妄想这一高置信度幻觉现象，通过多模型多任务的实证分析揭示其普遍性与顽固性，并探索检索增强生成和多智能体辩论等缓解策略，为提升模型可靠性提供方向。
摘要：
大型语言模型经常生成事实错误但看似合理的输出，即所谓的幻觉。我们发现了一种更隐蔽的现象——LLM妄想，定义为高置信度幻觉，即异常高置信度的错误输出，使其更难被检测和缓解。与普通幻觉不同，妄想以低不确定性持续存在，对模型可靠性构成重大挑战。通过在多个问答任务上对不同模型家族和规模的实证分析，我们表明妄想普遍存在且与幻觉不同。LLM在产生妄想时表现出更低的诚实度，且更难通过微调或自我反思来纠正。我们将妄想的形成与训练动态和数据集噪声联系起来，并探索了检索增强生成和多智能体辩论等缓解策略。通过系统地研究LLM妄想的性质、普遍性和缓解方法，我们的研究为这一现象的根本原因提供了见解，并概述了提高模型可靠性的未来方向。
英文摘要：
Large Language Models often generate factually incorrect but plausible outputs, known as hallucinations. We identify a more insidious phenomenon, LLM delusion, defined as high belief hallucinations, incorrect outputs with abnormally high confidence, making them harder to detect and mitigate. Unlike ordinary hallucinations, delusions persist with low uncertainty, posing significant challenges to model reliability. Through empirical analysis across different model families and sizes on several Question Answering tasks, we show that delusions are prevalent and distinct from hallucinations. LLMs exhibit lower honesty with delusions, which are harder to override via finetuning or self reflection. We link delusion formation with training dynamics and dataset noise and explore mitigation strategies such as retrieval augmented generation and multi agent debating to mitigate delusions. By systematically investigating the nature, prevalence, and mitigation of LLM delusions, our study provides insights into the underlying causes of this phenomenon and outlines future directions for improving model reliability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
