模型在想什么？通过模型内部状态分析理解大语言模型幻觉的“心理学”
What are Models Thinking about? Understanding Large Language Model Hallucinations "Psychology" through Model Inner State Analysis
AI总结：
针对大语言模型幻觉检测依赖外部信息源、延迟高的问题，该研究将模型推理内部状态引入幻觉检测，划分推理三阶段提取状态并分析其检测能力与优劣势，为理解幻觉成因提供新视角。
摘要：
大语言模型（LLM）系统生成有效且符合事实内容的能力不稳定，会导致幻觉生成。当前的幻觉检测方法严重依赖模型外部信息源（如RAG（检索增强生成））来辅助检测，因此会带来很高的额外延迟。近年来，LLM推理的内部状态已被广泛应用于提示注入检测等众多研究工作中。考虑到LLM内部状态的可解释性以及无需外部信息源的特点，我们将这类状态引入LLM幻觉检测任务。本文系统分析了推理前向过程中不同内部状态的表征特征，并全面评估了它们在幻觉检测中的能力。具体而言，我们将大语言模型的前向过程划分为理解、查询、生成三个阶段，并从这些阶段提取内部状态。通过分析这些状态，我们深入理解了幻觉内容产生的原因以及模型内部状态的变化过程。随后，我们将这些内部状态应用于幻觉检测，并开展了全面实验以探讨其优势与局限性。
英文摘要：
Large language model (LLM) systems suffer from the models' unstable ability to generate valid and factual content, resulting in hallucination generation. Current hallucination detection methods heavily rely on out-of-model information sources, such as RAG to assist the detection, thus bringing heavy additional latency. Recently, internal states of LLMs' inference have been widely used in numerous research works, such as prompt injection detection, etc. Considering the interpretability of LLM internal states and the fact that they do not require external information sources, we introduce such states into LLM hallucination detection. In this paper, we systematically analyze different internal states' revealing features during inference forward and comprehensively evaluate their ability in hallucination detection. Specifically, we cut the forward process of a large language model into three stages: understanding, query, generation, and extracting the internal state from these stages. By analyzing these states, we provide a deep understanding of why the hallucinated content is generated and what happened in the internal state of the models. Then, we introduce these internal states into hallucination detection and conduct comprehensive experiments to discuss the advantages and limitations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
