学会相信你的“直觉”：利用大语言模型的自我意识减轻幻觉
Learning to Trust Your Feelings: Leveraging Self-awareness in LLMs for Hallucination Mitigation
AI总结：
本研究发现LLMs具备内部知识状态自我意识但生成时易出现事实幻觉，提出融合知识探测与一致性检查的标注工具Dreamcatcher及知识反馈强化学习框架RLKF，有效提升模型事实性与诚实度及相关任务表现。
摘要：
我们评估了大语言模型（LLMs）识别并表达其内部知识状态的能力，这是对抗事实性幻觉、保障LLMs可靠应用的关键因素。我们观察到LLMs具备稳定的内部知识状态自我意识，知识探测实验的准确率超过85%。然而，LLMs在生成过程中往往无法表达其内部知识，进而引发事实性幻觉。我们开发了一款自动化幻觉标注工具Dreamcatcher，该工具融合知识探测与一致性检查方法，对事实偏好数据进行排序。我们以知识偏好作为奖励，提出了知识反馈强化学习（RLKF）训练框架，借助强化学习提升LLMs的事实性与诚实度。我们在多个模型上开展的实验表明，RLKF训练有效增强了模型利用自身内部知识状态的能力，提升了其在各类基于知识及与诚实度相关任务上的表现。
英文摘要：
We evaluate the ability of Large Language Models (LLMs) to discern and express their internal knowledge state, a key factor in countering factual hallucination and ensuring reliable application of LLMs. We observe a robust self-awareness of internal knowledge state in LLMs, evidenced by over 85% accuracy in knowledge probing. However, LLMs often fail to express their internal knowledge during generation, leading to factual hallucinations. We develop an automated hallucination annotation tool, Dreamcatcher, which merges knowledge probing and consistency checking methods to rank factual preference data. Using knowledge preference as reward, We propose a Reinforcement Learning from Knowledge Feedback (RLKF) training framework, leveraging reinforcement learning to enhance the factuality and honesty of LLMs. Our experiments across multiple models show that RLKF training effectively enhances the ability of models to utilize their internal knowledge state, boosting performance in a variety of knowledge-based and honesty-related tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
