相信我，我错了：尽管知道答案，大语言模型仍会自信地产生幻觉
Trust Me, I'm Wrong: LLMs Hallucinate with Certainty Despite Knowing the Answer
AI总结：
针对LLM存在的CHOKE现象（即模型本可正确回答问题，却因现实中可能出现的微小扰动而自信产生幻觉），研究证实其跨提示词、模型和数据集的一致性及与其他幻觉的差异，提出基于探测的缓解方法并验证其优于现有方法，为提升LLM安全性提供新视角。
摘要：
此前关于大语言模型（LLM）幻觉的研究将其与模型不确定性或知识不准确相关联。本研究定义并探究了一种独特的幻觉类型：模型能持续正确回答问题，但现实场景中可能出现的看似无关紧要的扰动，会使其以高置信度生成幻觉响应。我们将这种现象称为CHOKE（Certain Hallucinations Overriding Known Evidence，确定性幻觉覆盖已知证据），它在医学或法律等高风险领域尤为令人担忧，因为模型置信度常被用作可靠性的代理指标。我们发现CHOKE示例在不同提示词中具有一致性，存在于不同模型和数据集，且与其他幻觉存在根本差异。这种差异导致现有缓解方法在CHOKE示例上的表现比在一般幻觉上更差。最后，我们提出一种基于探测的缓解方法，在CHOKE幻觉上的表现优于现有方法。这些发现揭示了幻觉被忽视的一面，强调需要理解其起源并改进缓解策略以增强LLM安全性。代码可在https://github.com/technion-cs-nlp/Trust_me_Im_wrong获取。
英文摘要：
Prior work on large language model (LLM) hallucinations has associated them with model uncertainty or inaccurate knowledge. In this work, we define and investigate a distinct type of hallucination, where a model can consistently answer a question correctly, but a seemingly trivial perturbation, which can happen in real-world settings, causes it to produce a hallucinated response with high certainty. This phenomenon, which we dub CHOKE (Certain Hallucinations Overriding Known Evidence), is particularly concerning in high-stakes domains such as medicine or law, where model certainty is often used as a proxy for reliability. We show that CHOKE examples are consistent across prompts, occur in different models and datasets, and are fundamentally distinct from other hallucinations. This difference leads existing mitigation methods to perform worse on CHOKE examples than on general hallucinations. Finally, we introduce a probing-based mitigation that outperforms existing methods on CHOKE hallucinations. These findings reveal an overlooked aspect of hallucinations, emphasizing the need to understand their origins and improve mitigation strategies to enhance LLM safety. The code is available at https://github.com/technion-cs-nlp/Trust_me_Im_wrong .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
