大型语言模型中自动化幻觉检测的（不）可能性
(Im)possibility of Automated Hallucination Detection in Large Language Models
AI总结：
本文从理论上证明，仅用正确示例训练时，LLM 幻觉检测对多数语言集合不可行；但引入专家标注的正负例反馈后，对所有可数语言集合均可实现自动化检测，为 RLHF 等方法提供了理论支撑。
摘要：
自动化幻觉检测是否可能？在本文中，我们引入一个理论框架，用于分析自动检测大型语言模型（LLMs）所产生幻觉的可行性。受经典的语言识别 Gold-Angluin 框架及其近期由 Kleinberg 和 Mullainathan 对语言生成的改编的启发，我们研究一个算法——该算法在从未知目标语言 $K$（从可数集合中选取）中抽取的示例上训练，并可访问一个 LLM——是否能够可靠地判断该 LLM 的输出是正确的还是构成幻觉。首先，我们建立了幻觉检测与经典语言识别任务之间的等价性。我们证明任何幻觉检测方法都可以转换为语言识别方法，反之，解决语言识别的算法也可以适配用于幻觉检测。鉴于语言识别固有的困难，这意味着如果检测器仅使用来自目标语言的正确示例进行训练，那么对于大多数语言集合而言，幻觉检测在根本上是不可行的。其次，我们表明，使用专家标注反馈，即同时使用正例（正确陈述）和负例（明确标注为不正确的陈述）来训练检测器，会极大地改变这一结论。在这种增强的训练机制下，自动化幻觉检测对于所有可数语言集合都变得可行。这些结果凸显了专家标注示例在训练幻觉检测器中的关键作用，并为基于反馈的方法（如基于人类反馈的强化学习 RLHF）提供了理论支持，这些方法已被证明对于 LLM 的可靠部署至关重要。
英文摘要：
Is automated hallucination detection possible? In this work, we introduce a theoretical framework to analyze the feasibility of automatically detecting hallucinations produced by large language models (LLMs). Inspired by the classical Gold-Angluin framework for language identification and its recent adaptation to language generation by Kleinberg and Mullainathan, we investigate whether an algorithm, trained on examples drawn from an unknown target language $K$ (selected from a countable collection) and given access to an LLM, can reliably determine whether the LLM's outputs are correct or constitute hallucinations. First, we establish an equivalence between hallucination detection and the classical task of language identification. We prove that any hallucination detection method can be converted into a language identification method, and conversely, algorithms solving language identification can be adapted for hallucination detection. Given the inherent difficulty of language identification, this implies that hallucination detection is fundamentally impossible for most language collections if the detector is trained using only correct examples from the target language. Second, we show that the use of expert-labeled feedback, i.e., training the detector with both positive examples (correct statements) and negative examples (explicitly labeled incorrect statements), dramatically changes this conclusion. Under this enriched training regime, automated hallucination detection becomes possible for all countable language collections. These results highlight the essential role of expert-labeled examples in training hallucination detectors and provide theoretical support for feedback-based methods, such as reinforcement learning with human feedback (RLHF), which have proven critical for reliable LLM deployment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
